tag:blogger.com,1999:blog-4544550922473633662024-03-05T12:25:37.092-08:00Chris' Coding FunI'll be talking about all things Rails, Python, Ruby and Java among other topics in this blog. I also enjoy a bit of system programming, so my fun with WebOS, Win32 and Solaris may show up here too.TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.comBlogger37125tag:blogger.com,1999:blog-454455092247363366.post-40442435153247802462018-08-09T00:16:00.000-07:002020-02-11T00:19:21.832-08:0064bit Azure App Service for DotNet CoreJust a note. Azure App services do not directly support 64bit dotnet core applications.<br />
<br />
<br />
https://blogs.msdn.microsoft.com/webdev/2018/01/09/64-bit-asp-net-core-on-azure-app-service/<br />
<br />
TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-50563529546233148802016-01-05T07:39:00.002-08:002016-01-05T07:39:30.634-08:00Its Happening! Its really Happening! ^_^So, big news. The Oculus/Facebook folks just announced that all kickstarter backers who got a unit, will get a commercial unit! As, an original backer of the Oculus (and one of the backers who was actually ok with the facebook purchase), I am extremely happy, and extremely proud/humbled that I was a part of helping VR come back. <div>
<br /></div>
<div>
Big Thank you to the folks at Oculus!</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<blockquote class="tr_bq" style="background-color: white; border: 0px; color: #0f2105; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 16px; font-stretch: inherit; line-height: 22.4px; margin-bottom: 20px; padding: 0px; vertical-align: baseline;">
As a small token of our appreciation for your support, all Kickstarter backers who pledged for a Rift development kit will get a free Kickstarter Edition Oculus Rift! And like all Rift pre-order purchasers, you’ll receive a bundled copy of Lucky’s Tale and EVE: Valkyrie.</blockquote>
<br />
<blockquote class="tr_bq" style="background-color: white; border: 0px; color: #0f2105; font-family: 'Helvetica Neue', Helvetica, Arial, 'Liberation Sans', FreeSans, sans-serif; font-size: 16px; font-stretch: inherit; line-height: 22.4px; margin-bottom: 20px; padding: 0px; vertical-align: baseline;">
To claim your free Rift, fill out the survey you’ll receive from Kickstarter before February 1, 2016 and provide a shipping address that is located in one of the 20 countries Rift is launching in. For those who don’t live in one of those 20 countries, we’re working on an alternative, and you can let us know your preference in the form.</blockquote>
<br /></div>
TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-6095897072301098252015-06-08T06:43:00.000-07:002015-06-08T07:47:39.989-07:00signalr and friends<h3>
What have I been up to?</h3>
<div>
Currently I am really into Microsoft's SignalR technology. It has me thinking about things like realtime applicatoin configurations as well as realtime dashboards with decentralized data points. Right now you have things like splunk and such which are great for historical processing of data, etc but what if you want realtime monitoring of an application and also communicate configuration to that application? </div>
<div>
<br></div>
<div>
There are some great examples out there of this stuff, So i am going to dive in and hopefully have some code up on bitbucket or github!</div>
<div>
<br></div>
<div>
Also I am a bit interested in bringing a more efficient transport mechanism to SignalR like <span style="background-color: white; color: #545454; font-family: arial, sans-serif; font-size: x-small; line-height: 14.5600004196167px;">MessagePack</span><span style="background-color: white; color: #545454; font-family: arial, sans-serif; font-size: x-small; line-height: 14.5600004196167px;"> </span>or Protocol Buffers or even an elder statesmen like hessian!</div>
<div>
<br></div>
<div>
<br></div>
TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-29981909885034507942014-02-01T19:11:00.000-08:002014-02-01T19:11:41.444-08:00Finding Libc<br />
<br />
So, as you know, I've been tooling around with the LLVM IR syntax and the Gold Parser framework. I have the scaffolding for the CIL conversion 90% done, and I am almost ready to parse the "Hello world" code found in the LLVM documentation. There is just one catch. In the Hello world, there is a call to an external standard io library function called <b>puts()</b>. Those fimilar with C/C++ would know that you include the standard header, #include <stdio> or #include <stdio.h> and you magically gain access to this function.<br />
<br />
Porting that specific line into CIL would be easy, i'd just replace any puts call with a WriteLine call included in the standard System Library. This is fine to get the sample code running, but it is not a solution going forward. I need to get a list of basic libc commands and rewrite them to work with the System library in the CLI.<br />
<br />
<br />
DotGnu Project Resurrected?<br />
<br />
Years ago (around 2003-2004), the GNU community was really interested in CIL (at the time called MSIL) and its potential to have a common assembly language across disparate systems. One very real problem with *inx development is the fragmentation of libraries. C# was going to solve that problem (I posit that LLVM IR has effectively solved the problem....but thats my opinion). Many open source projects rose to the occasion of "Freeing" .net and MSIL from the hands of Microsoft. From that rush, we have great frameworks like Mono (and i believe Microsoft felt massive pressure, and decided to release most of the .net framework source under the MS-PL, a fairly open license). Lots of these frameworks died or became less interesting, especially as LLVM finished CLANG, etc.<br />
<br />
Fast forward to today, and my need to basically port large portions of libc to .net. It seems DotGnu has already done this. Its under the GPL, but for my purposes, I dont care. This is a proof of concept and once i've finished getting things working, there could be a discussion of the merits of using MIT/X11 based licenses, etc. DotGnu has a module called pnetC, that includes a fairly feature complete port of libc to CIL.<br />
<br />
Now that I have a CIL port of libc, its a matter of getting a list of all libc functions and adding the namespace to the call of that method. I'll test this functionality with the puts() function in the example code. This is a much better use of time, then just getting WriteLine to work (though i still may do that, just to get a clean parsing structure finished in the next few hours).<br />
TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-35585806507681048692014-01-28T21:12:00.000-08:002014-01-28T21:13:12.089-08:00Goodbye ZFS?<br />
<h3>
ZFS my one true love</h3>
I've used ZFS at home since about 2007. Thats a long time for a technology. I have used it through Solaris 9, 10, 11, Nenenta, Illumos, at al. It is an amazing feat of engineering. Dtrace is another little thing i use, from time to time, on Solaris as well...but not so much in recent times.<br />
<br />
<br />
<h3>
Time for an annulment?</h3>
While I could say its been all roses, it hasnt. There have been issues with running third party software on solaris. FFMPEG was a beast in the early days to get compiled an running. I used my storage array for many things, and transcoding was one of them. I wanted to move away from running things like tversity on my desktop. I also wanted to get away form development on my desktop, so I started moving toward VMs for everything, which turned out to be the best thing ever....except for a few issues....USB support was shabby for awhile on virtualbox so deploying software onto a device was difficult (to say the least). Things have gotten better. I attempted to mitigate this by running an experiment. Trying to get iSCSI boot working on Windows Vista (and later, attempted with Win 7). I was partially there, but later found out my pool, which was designed for long term, large file storage...wasnt cut out for the random I/O needed for an OS to run adequtely (I used a single vdev!!!).<br />
<br />
<h3>
Commodities are a beautiful thing</h3>
I am at a crossroads now. SSDs are cheap. Ram is cheap, Spinning Storage is cheap. I can now have a high performance development environment booted straight from my storage array. an environment that supports zfs snapshots and a host of other ZFS features (de-dup, etc).<br />
<br />
Why the long story about virtualization and Solaris? Well, i am looking to change my core OS. One option is to run ESXi Hypervisor and enjoy ZFS, but allow for me to run other virtual instances... I would no longer have to store them locally and could have them all centrally managed.<br />
<br />
As I went down this road, I remembered that Windows has Hyper-v...and that RemoteFX for Windows Server 2012 allows for full local GPU use. Not only that, Microsoft has released a ZFS clone of sorts with ReFS and Storage Spaces. It has self healing and many other great features that I use on ZFS. Its missing "Free" Snapshots and its still unproven with large installations.<br />
<br />
<br />
To Be continued!!<br />
<br />
<br />
<br />
<br />
<br />TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-13613040282211152812014-01-28T21:05:00.000-08:002014-01-28T21:05:20.869-08:00Quick Progress!<div>
<br /></div>
<div>
<h3>
Parsing for fun and profit</h3>
</div>
I remember creating parsers to be a bit harder. I tried using Antlr when doing the JavaFX DLR project and failed at it miserably. It felt messy. I couldnt get a feel for how to structure the document, etc. I was drawn to the Irony parser framework because declaring the grammar in C# somehow felt (at the time) cleaner and it seemed to allow for more expressive statements and character logic.<br />
<div>
<br /></div>
<div>
I am not sure if it was my previous experience with parsing and Antlr (or Irony), or how clean and expressive the GOLD parser framework is, but I've been able to create a working parser in just a few days of coding. And by working, I mean it parses and understands valid LLVM IR straight from the project's documentation: </div>
<div>
<script src="https://gist.github.com/techsaint/8681959.js"></script><br /></div>
<div>
I actually implemented a a lot more operations and keywords needed to parse the example. It was just easier that way. I am going to now start building an AST. I've decided to be a bit aggressive because I think the AST will drive some Grammar refactoring. I'd like to do the refactoring sooner rather than later. Right now the grammar is fairly small and I'd like to refactor using the AST as a guide. (Think BDD, but the behavior is defined for me, in CIL...i just need to start outputing CIL to validate/test!). This will also allow me to structure the document better and add more comments for posterity.<br />
<br />
So my next steps:<br />
<br />
<ol>
<li>Create an AST</li>
<li>Generate IL with the AST</li>
<li>Create a running and valid CIL</li>
<li>Refactor GOLD Grammar</li>
<li>Add missing Operations and Instructions to Grammar</li>
<li>Add respective AST logic</li>
<li>Find small C (or C++) library to test Grammar and IL generator (lzip is a contender)</li>
<li>Find a libc to bootstrap: ulibC or newlib are contenders</li>
</ol>
<br />
<br />
I am hoping to have 1-3 done in the next few days. 4-6 soon after that. I have very little experience with bootstrapping, so i'll probably have a look at the emscripten project as a guide. For those who dont know (or if i am using the wrong terminology), basically a general C/C++ library will use standard calls (syscalls, printf, cout, etc, etc) out to what is known as libc (though thats a specific library, there are many implementations). I will need to create a shim routes those calls to the respective mscorelib class/method or find a sutible replacement library on the .net framework (take SDL or Opengl for example...)<br />
<br />
<br />
<br />
Note to those trying to create an grammar for LLVM IR. you see that c"<String>" notation? Its not mentioned ANYWHERE in their documentation. I am assuming it is short hand to create a character array from the given string. I've found a few other little syntax surprises hidden in the example code of the documentation. Its not too bad...yet. Because I am not worrying about my parser validating the syntax, i think I may even be able to ignore some of those surprises. </div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-15324348302190693442014-01-28T07:48:00.000-08:002016-01-05T17:08:28.642-08:00<i>So I've again been bitten by the Transcoding/transpiling bug...Doh!</i><br />
<br />
<h3>
My history with Transcoding</h3>
A few years back (around mid-2007), I was enamored with a new language on the block, JavaFX (now vala). Alas, I had moved on from the Java platform to the .Net platform during that time, and Silverlight just hit the stage (and WPF). I was more of a fan of the declarative and expressive nature of JavaFX than I was of XAML, so I looked into porting JavaFX script to the .Net platform (and later the DLR). I halted my work in 2008-2009, as JavaFX script was dying, and the Parser technology I was using (a great C# based parser called Irony) was giving me issues when integrating with the DLR.<br />
<br />
<br />
<h3>
The LLVM Project</h3>
<br />
I've been a fan of the LLVM project since about 2009. I have also watched its MSIL backend go into disrepair and interest in C# as part of the vmkit wane. I answered some stack overflow questions in 2011 about this http://stackoverflow.com/questions/5054938/how-to-translate-msil-to-llvm-li/8272816#8272816<br />
<br />
Around 2011 I was looking at resurrecting the MSIL backend and decided while it would be of great use to the .net community (FFMPEG as a .net DLL would do wonders alone), it was something I couldnt commit to. And as of today, the MSIL backend has been completely removed in the 3.x branch of LLVM, and would need a major commitment to get it back up to usable levels. (Which one day, I may contribute to..)<br />
<br />
<h3>
LLVM IR....a better CIL</h3>
<br />
So fast forward to now. MSIL is now referred to as CIL and the LLVM project's intermediate language LLVM IR is amazing. Now I am interested in parsing LLVM IR and creating CIL. Apparently the Mono project has this going the other way, they have CIL code that gets transcoded into LLVM IR.<br />
<br />
<br />
<h3>
Of Parsers and Men</h3>
<br />
<br />
I've decided to create a non validating parser for LLVM IR using the GOLD Parser project (I will assume wellformatted IR for this experiment). And then from there, build an AST that outputs CIL!TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-44054235602154291632013-05-31T11:03:00.000-07:002013-05-31T11:04:11.202-07:00Fun with the CloudUpdate!<br />
<div>
<br /></div>
<div>
I've been busy these last few months. I've Worked on porting Mono to the BB10 platform. worked on porting Google's Snappy compression to ZFS and a few other little side projects.</div>
<div>
<br /></div>
<div>
Recently I came across an intriguing open source project, GamingAnywhere. GA basically a private cloud solution for Cloud Gaming services like Gaikai or OnLive. Cloud gaming has always interested me. What is interesting about private cloud gaming is that you can use your general purpose PC as a gaming hub for your house. You can have a fairly nice Mid to high end PC and, in theory, not have to care about how it fits in with your entertainment system etc.<br />
<br />
When you set up a GamingAnywhere server, it pipes your full desktop, or a specific named window'ed application to a GamingAnywhere client. It currently uses lag free x264 settings, at 24fps (Which if course brings up bandwidth and latency issues...the same issues that OnLive et al, have been working on). Currently both the client and server support windows and Linux.<br />
<br />
More to come on this front, as I am still diving in. </div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-66665258862754579412012-09-27T21:12:00.000-07:002012-09-27T21:12:45.218-07:00BB10 and MonoI am back, with my usual mono antics! RIM recently had a conference and showed off more of BB10. I have decided I will attempt (albiet in my spare time) to use C# to create a Cascade application. I have a few ideas how i'll accomplish this. I'll post more info as time goes on.TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-24991686735972068702012-05-28T18:21:00.000-07:002012-05-29T03:15:03.968-07:00Tracking down Signal codesOk. I've been trying to tracked down the error's meaning. specifically SIGSEGV code=2. I found it listed in the header file located at <span style="background-color: #e5e5e5; color: #656565; font-family: 'Trebuchet MS', 'Lucida Grande', Arial, sans-serif; font-size: 13px; line-height: 20px; text-align: left;"> </span><span style="background-color: #e5e5e5; color: #656565; font-family: 'Trebuchet MS', 'Lucida Grande', Arial, sans-serif; font-size: 13px; line-height: 20px; text-align: left;">sys/siginfo.h</span> :<br />
<br />
* SIGSEGV codes (si_addr == address of faulting memory reference)<br />
*/<br />
#define SEGV_MAPERR 1 /* Address not mapped */<br />
#define SEGV_ACCERR 2 /* No permissions */<br />
#define SEGV_STKERR 3 /* ?? Stack exception */<br />
#define SEGV_GPERR 4 /* ?? General protection */<br />
#define SEGV_IRQERR 5 /* ?? Interrupt handler fault */<br />
#if defined(__EXT_QNX)<br />
#define NSIGSEGV 5<br />
#endif<br />
<br />
It seems to be SEGV_ACCERR. No permissions. Very interesting. I am not sure what it is trying to access, or why i dont have permissions as a developer. Hmmm... <br />
<br />
<br />
Update: Found addr2line. seems like there are some errors in aot-compiler.c, and some errors in mini-arm.c. I may just do a disable-aot and focus on the mini-arm.c issue...<br /><br />
<div>
<br /></div>TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-8456215899991729222012-05-28T07:03:00.002-07:002012-05-28T07:12:37.312-07:00some SSH on the playbookI was able to use blackberry-connect to open up ssh access on my playbook. SCP'ed my mono install and received this error message:<br />
<br />
<br />
Process 10866804 (mono) terminated SIGSEGV code=2 fltno=11 ip=000bfde0<br />
Memory fault (core dumped)<br />
<br />
Not sure what it means. I did some strange sig hacks, so I may start there. I wonder if i could use gdb to check out the core dump.<br />
<br />
When i run Mono -V i get:<br />
<br />
<br />
$ ./mono -V <br />
Mono JIT compiler version 2.10.9 (mono-2-10-9/256094a Mon May 28 09:15:52 EDT 2012)<br />
Copyright (C) 2002-2011 Novell, Inc, Xamarin, Inc and Contributors. www.mono-project.com<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>TLS: normal<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>SIGSEGV: normal<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>Notification: Thread + polling<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>Architecture: armel,vfp<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>Disabled: none<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>Misc: softdebug<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>LLVM: supported, not enabled.<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>GC: none<br />
<div>
<br /></div>
<div>
I wonder if SIGSEGV has anything to do iwth it. Also, not sure about the architecture. is armel,vfp correct?</div>
<br />
<div>
<br /></div>TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-87253509527188961002012-05-27T23:51:00.002-07:002012-05-27T23:51:39.639-07:00A nod to Linux Mint and Virtual BoxI have went through quite a few linux distros in my time, from Red Hat to Slackware, knoppix, to Mandrake, Caldera, Debian, Gentoo (one of my favorite) to recently Ubuntu (I am also an AfterStep fan..lol). I chose Ubuntu as my main development platform of choice a few years ago because of its debian connections, the apt-get repository system is well supported, and third party repositories are easily found. (I found, while i loved Gentoo, and portage was amazing....it was slow on getting a system up and running).<br />
<br />
For the past few months I have been trying out Linux Mint (specifically 12, though now 13 is out). I have to say I am really impressed at how clean the interface is. After Ubuntu jumped the shark with their plasma interface I was longing for a simple gnome based (though i'd even go for kde) interface. I just want to jump in, install any supporting software, hack, tinker, coded, build, commit and jump out.<br />
<br />
Just some background, A few years ago I decided to get rid of all of my dev PCs (around 4) and just focus on one uber PC (and one storage array), with high end specs, and do all my development (at least as much as technically possible) in Virtual Machines. So I end up with a quad core i7, 24GB of ram, 3TB HD local, and a 10TB raided storage array. I also decided that all my development would take place using source control, either git or mercurial, and stored on my storage array and/or github/bitbucket.<br />
<br />
My experience with Ubuntu was getting progressively worse. I ended up fighting the installation and the interface. Linux Mint is clean, fairly light, and fast. I need a Linux that can install in minutes. My profile is as such, when i want to tinker, I spin up a clean virtual machine (usually by installing from an ISO), give it 1-2GB of ram, and a single core of CPU power, 100GB of dynamic HD space (meaning it only uses the space needed, but tops out at 100GB, just in case of any run away processes, etc). This allows me to tinker without other software getting in the way, and my main desktop is clean for work. I have actually even toyed with the idea that any web browsing and IMing should be done in a VM.<br />
<br />
<br />
Note: I only do this for linux/solaris development, as windows is a bit harder to deal with. One day we'll have repositories and package managers like apt-get on windows. apt-get allows me to pull down a dev environment within 20 mins. Latest patches. the works. On windows, from bare metal to working VS install (with all the nuget tooling) takes hours, and thats not including any servers needed to install. Windows 7 is much better though...TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-48234461003508059322012-05-27T22:53:00.001-07:002012-05-27T22:53:26.577-07:00PlayBook + Mono: Coming soonI spent part of the weekend cross compiling Mono on RIM's playbook/QNX platform. I will be uploading the code to github (or bitbucket) and creating a patch. I purposely used the mono-2-10.8 branch, so there would be no need for messing around with changesets. doing a "git checkout mono-2-10-8" and then applying my patch will work. There are some limitations, such as no sgen. Also i commented out a timezone function. I'll post a full list once i actually upload the code.<br />
<br />
<br />
I am actually attempting to run the new mono build on the playbook, but I am having trouble. When i did this with WebOS, it was much easier. I just SCP'ed the build to my WebOS device, and ran some C#. I am unsure how to best go about this on the playbook. Time for some more tinkering.TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-1963662725895131282012-05-20T14:35:00.001-07:002012-05-20T14:35:07.716-07:00Free ebooks: Great content from Microsoft Press that won’t cost you a penny - Microsoft Press - Site Home - MSDN BlogsFree eBooks from Microsoft.<br />
<br />
I grabbed all of them, but I am most interested in the Windows Phone 7 and SQL Server 2012 books.<br />
<br />
<a href="http://blogs.msdn.com/b/microsoft_press/archive/2012/05/04/free-ebooks-great-content-from-microsoft-press-that-won-t-cost-you-a-penny.aspx">Free ebooks: Great content from Microsoft Press that won’t cost you a penny - Microsoft Press - Site Home - MSDN Blogs</a>: <br />
<br />
<br />TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-87245282774828911922012-05-19T10:38:00.000-07:002012-05-27T10:41:19.839-07:00Haiku + Mono: Almost thereAfter trying to compile and install mono 2.10.8 on the latest builds of Haiku, its apparent that Haiku is now getting some Mono love. Its not 100% complete, but while spelunking through source I saw specific __HAIKU__ #ifdef tags, which didnt exist when i first started looking into getting mono working.<br />
<br />
It looks like by the next alpha or release candidate mono will be up and running. Its doubtful my endeavors really helped. Though going forward I will be, at the very least, publishing the majority of hacks so they may be used as information for patches going forward. I will also attempt (where i see the need) to get patches committed into Mono (and any other projects/tech that I hack regularly).<br />
<br />
I think the knowledge gained from hacking Mono/WebOS, Mono/Haiku and ZFS/Snappy (among others) will allow me to start creating useful patches. I've learned a lot about autoconf, automake, et al.TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-54392932954387577202012-05-18T07:32:00.000-07:002012-05-20T07:40:59.673-07:00Haiku and MonoOne of my many side tinkerings has been porting Mono to different platforms. I've<a href="http://code.google.com/p/webosdotnet/"> ported Mono successfully to WebOS 2.0</a> (which was instantly broke when 3.0 came out, and I have not looked into it further). I've also been looking at porting it to the Playbook OS. One of the platforms that I attempted and failed was Haiku.<br />
<br />
The failure was a combination of things. It was about 2-3 years ago, and Mono had some glib dependencies that were an issue, as well as libgdiplus. And then on the haiku side, libiconv, pthreads and POSIX signal.h wasnt available. I used <a href="http://ports.haiku-files.org/wiki/dev-lang/mono/2.x/1">this wiki entry</a> to help guide me.<br />
<br />
I actually successfully compiled mono on haiku 2 years ago, after some header file hacks, some edits on the mono side, etc. I deemed it way way too neutered to do move forward, and dropped the project. Back around July 2011 alpha 3 was released (i was doing my initial stuff on alpha 2), and after checking the <a href="http://ports.haiku-files.org/wiki/CommonProblems">Haiku Porting guide</a> I figured, "why not try?", so I spun up a Haiku alpha 3 VM. Amazingly libiconv and pthreads were fixed, and mono now had a working embedded glib option. I still ran into the signal.h wall. I decided not to add my header hacks, since I know very little about signal handling from the kernel side, and wasnt prepared to push for kernel changes in haiku. Looking back, it may have been useful to catalog my tinkerings for the masses, but oh well.<br />
<br />
Fast forward to a few days ago, I noticed <a href="http://dev.haiku-os.org/wiki/SignalSupportRevision">this wiki entry</a> and specifically I noticed the bug entry for "<span style="background-color: white;"><span style="font-family: Verdana, Arial, 'Bitstream Vera Sans', Helvetica, sans-serif; font-size: x-small;">Support for realtime signals is missing" had the numbers crossed out. Come to find out, after a little bit more digging, that </span><a href="http://dev.haiku-os.org/ticket/1935" style="font-family: Verdana, Arial, 'Bitstream Vera Sans', Helvetica, sans-serif; font-size: 13px;">ticket</a><span style="font-family: Verdana, Arial, 'Bitstream Vera Sans', Helvetica, sans-serif; font-size: x-small;"> was fixed. The signal issue was no more, but it was a few months <i>after</i> the alpha 3 release! On a quick side note, the haiku project has this excellent source control/build system. they have nightly builds that get packaged up as anyboot/ISO/tar.gz deployments. This allows any developer to instantly test code without hunting down instructions on how to build an ISO or USB boot device. Developers on Haiku can focus on whats important, the code. </span></span><br />
<span style="background-color: white; font-family: Verdana, Arial, 'Bitstream Vera Sans', Helvetica, sans-serif; font-size: 13px;"><br /></span><br />
<span style="background-color: white; font-family: Verdana, Arial, 'Bitstream Vera Sans', Helvetica, sans-serif; font-size: 13px;">Anyway, I quickly downloaded the latest nightly, which includes the signal fixes, and had it up and running in 30 mins. I am now poking around, once again, to see if i can get mono to compile....this time without any haiku changes. I would like to be able to create a patch for the community.</span>TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-12079642878066280712012-05-12T18:41:00.001-07:002012-05-12T18:58:20.856-07:00Snappy TestingAlright. I've finally got around to doing some Snappy testing. I've gotten some amazing results, but also noticed a bug. My current code seems to hang after a large amount of compression. It specifically hangs when i am doing an iozone benchmark. The 500MB pass (I set it to 4GB). But before I ran iozone, i did some paliminary testing vs lzjb, and I have to say, the compression performance is higher with Snappy. I was able to get a 23.14 compression ration instead of the 15.22 ratio from lzjb, with similar cpu usage.<br />
<br />
I suspect this has to do with the snappy_env_init method in the snappy-c library i found. It seems fishy having to initializing the environment for every compression event. I may look into it further but it does not seem very scaleable. and given that ZFS compression works at the block level, and not the file level, there are a lot of compression events happening for a single file (instead of one event, compressing a 400MB file...it would be 400 events with a zfs file sysem that is allocated using 4k blocks).<br />
<br />
Anyway, I may or may not delve further into this. I think I could actually call this a finished project. Its in a usable state. Anyone can grab it from github and run snappy with zfsonlinux. Snappy could be a real contender in the ZFS space. I might even think about adding other algorithms to zfs. 7z, bzip and ARJ would be fun.TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-74293662654389333372012-05-07T09:07:00.001-07:002012-05-08T11:22:44.484-07:00Interesting technologyFound an interesting project called <a href="http://vertx.io/">vertx.</a>x I might end up trying to port it to the CLR using the technology that the folks over at the mono-project used for porting android's java layer to C#<br />
<br />
<br />
<br />TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-14242231732406969442012-05-06T22:06:00.000-07:002012-05-08T19:40:39.926-07:00ZFS + Snappy = DoneI've completed my first zfs + snappy build. Its up on github at <a href="https://github.com/techsaint/zfs-snappy">https://github.com/techsaint/zfs-snappy</a>. I tried to test it on my Virtualbox instance, but it doesn't look like there is very good VDI support for zfs. It seems to freeze when i try to write to a file system. I tried with the non snappy version too, just in case I mucked up something, but the vanilla version also froze. Now that I have a compatible version, I will test it on actual hardware, instead of in a VM.<br />
<br />
This build actually has snappy baked into zfsonlinux using the <a href="https://github.com/andikleen/snappy-c">snappy-c</a> library I found. If this route works, I will look at using the snappy-c port in an illumos build. I have all the changes out on bitbucket, and I did spend a few months learning the opensolaris/illumos build files....TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-11896966030029860622012-05-06T05:29:00.001-07:002012-05-06T05:29:53.569-07:00AutoConf and M4 blissJust wanted to make a statement that the CLI tooling for building software in linux (and really any *nix) has changed dramatically. In my mind, configure and make were all black boxes and it was very very hard to actually effect them as an independent developer once they were baked.<br />
<br />
This is not the case anymore. I was able, with autoconf, automake, and m4 templates (and of course some C++ glue) to actually integrate two libraries (zfs and snappy-c) that know nothing about each other. This is a testament not just to the auto tooling, but the level of maturity of a lot of open source software out there. While make is still technically a black box (yes, make scripts are simple in concept, fine...but go look at Illumos makefiles and we'll talk...), it may be ok for make to stay in the shadows...like some x86 assembly that gets generated on the fly...<br />
<br />
I just might, after this zfsonlinux + snappy release, revisit illumos. But from what I recall of the original opensolaris documentation (which illumos referenced early on), you edit Makefiles directly, and basically need a Solaris Kernel expert to figure out how to add kernel level features of any large magnitude.TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-73500207175748977652012-05-05T07:04:00.000-07:002012-05-05T07:04:07.260-07:00Quick UpdateJust wanted to give a quick update. On the ZFS front i have really made inroads. I now have to decide whether to create a stand alone snappy kernel module (which could assist folks outside of my little tinkerings) or roll snappy into zfsonlinux, so i can do some benches, and see if its a viable alternative to the compression included in zfs. I am leaning toward kernel module, but I've found a project called snappy-c that is a completely C port of snappy. This will make option #1 super easy.<br />
<br />
<br />
On the tinkering front,<a href="http://blog.xamarin.com/2012/05/01/android-in-c-sharp/"> Xamarin has ported the java layer of android to C#</a>. I have already gotten my hands on the source. Planning to put it on a phone or tablet soon.TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-87204115450700904182012-05-02T06:33:00.001-07:002012-05-02T06:33:17.672-07:00Snappy Paydirt!I didnt expect to have the time, but I really became interested in answering the question of how easy working with the linux port of ZFS would be. I am here to say that I got everything compiled and installed last night. I am now trying to track down an shared library file that doesnt seem to be found. It exists, but my guess would be that the zfs tool is looking for it in /usr/lib instead of /usr/local/lib. The specific error is:<br />
<br />
<br />
<blockquote class="tr_bq">
/home/techsaint/files/zfs-project$ zpool<br />
zpool: error while loading shared libraries: libnvpair.so.1: cannot open shared object file: No such file or directory</blockquote>
<div>
<br /></div>TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-26823939727815916482012-05-01T08:26:00.002-07:002012-05-01T08:26:54.343-07:00ZFS and Snappy ReduxMy ZFS/Snappy project was put on hold, because of technical issues and lack of understanding (and documentation) of how to integrate a library into an Illumos build. I may have found a compromise.<br />
<br />
The <a href="http://zfsonlinux.org/">zfs on linux project</a> has become a bit more mature. I may switch up and use linux to create a snappy proof of concept. I understand linux build structure much more than open solaris build structure, and the things i dont understand are fairly well documented. Because<a href="https://github.com/zfsonlinux/zfs"> zfsonlinux is on github</a>, I'll most likely be using Github and forking the project.<br />
<br />TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0Waltham, MA, USA42.3764852 -71.235611342.3295647 -71.3145753 42.423405699999996 -71.1566473tag:blogger.com,1999:blog-454455092247363366.post-14995941513032349142012-01-10T07:30:00.000-08:002012-01-10T07:30:43.403-08:00Documenting an SSIS packageI've been doing SSIS since it was called DTS back in the old'en days. I enjoyed using it with SQL Server 2000. One thing I always had trouble with was actually documenting the darn thing. It actually leads into a more existential issue with documenting these Workflow type systems, as well as these newer cloud based systems (take Microsoft's data explorer project, etc).<br />
<br />
I am leaving my current job and I had some SSIS work that wasnt well documented. I do see there are some commercial tools like BI Documenter <a href="http://pragmaticworks.com/Products/Business-Intelligence/BIDocumenter/Default.aspx">http://pragmaticworks.com/Products/Business-Intelligence/BIDocumenter/Default.aspx</a> But it seems I'll just fire up an old fashion Word document for now, and embed screenshots.TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0tag:blogger.com,1999:blog-454455092247363366.post-62313761361676338782011-12-19T09:31:00.000-08:002011-12-19T09:34:33.639-08:00Small Snappy + ZFS breakthroughWoo!. Seems everything compiles perfectly. I had to copy the snappy libraries into the closed binary section of the build. Also, i had to build a 64bit and 32bit version of snappy (a bit difficult). I ran into a libtool bug (yes, another road block) when trying to compile the 64bit version of snappy.<br />
<br />
<br />
Right now, I use iscsi with virtualbox. So its difficult to import that into a different VM product (most desktop VM products dont support iSCSI booting).. I think i will copy over my illumos-gate directory to a now OpenIndiana installation, using a virtualHD, instead of iscsi. Then i should be able to import it into VMware or VirtualPC...and avoid the virtualbox bug I am experiencing with the latest illumos build.<br />
<br />
<a href="https://bitbucket.org/techsaint/zfssnappy">https://bitbucket.org/techsaint/zfssnappy</a>TechSainthttp://www.blogger.com/profile/18152427463450821072noreply@blogger.com0