Digital Wasabi

ZFS my one true love I've used ZFS at home since about 2007. Thats a long time for a technology. I have used it through Solaris 9, 10, 11, Nenenta, Illumos, at al. It is an amazing feat of engineering. Dtrace is another little thing i use, from time to time, on Solaris as well...but not so much in recent times. Time for an annulment? While I could say its been all roses, it hasnt. There have been issues with running third party software on solaris. FFMPEG was a beast in the early days to get compiled an running. I used my storage array for many things, and transcoding was one of them. I wanted to move away from running things like tversity on my desktop. I also wanted to get away form development on my desktop, so I started moving toward VMs for everything, which turned out to be the best thing ever....except for a few issues....USB support was shabby for awhile on virtualbox so deploying software onto a device was difficult (to say the least). Things have gotten better. I ...

Update! I've been busy these last few months. I've Worked on porting Mono to the BB10 platform. worked on porting Google's Snappy compression to ZFS and a few other little side projects. Recently I came across an intriguing open source project, GamingAnywhere. GA basically a private cloud solution for Cloud Gaming services like Gaikai or OnLive. Cloud gaming has always interested me. What is interesting about private cloud gaming is that you can use your general purpose PC as a gaming hub for your house. You can have a fairly nice Mid to high end PC and, in theory, not have to care about how it fits in with your entertainment system etc. When you set up a GamingAnywhere server, it pipes your full desktop, or a specific named window'ed application to a GamingAnywhere client. It currently uses lag free x264 settings, at 24fps (Which if course brings up bandwidth and latency issues...the same issues that OnLive et al, have been working on). Currently both the cli...

Alright. I've finally got around to doing some Snappy testing. I've gotten some amazing results, but also noticed a bug. My current code seems to hang after a large amount of compression. It specifically hangs when i am doing an iozone benchmark. The 500MB pass (I set it to 4GB). But before I ran iozone, i did some paliminary testing vs lzjb, and I have to say, the compression performance is higher with Snappy. I was able to get a 23.14  compression ration instead of the 15.22 ratio from lzjb, with similar cpu usage. I suspect this has to do with the snappy_env_init method in the snappy-c library i found. It seems fishy having to initializing the environment for every compression event. I may look into it further but it does not seem very scaleable. and given that ZFS compression works at the block level, and not the file level, there are a lot of compression events happening for a single file (instead of one event, compressing a 400MB file...it would be 400 events with...

I've completed my first zfs + snappy build. Its up on github at  https://github.com/techsaint/zfs-snappy . I tried to test it on my Virtualbox instance, but it doesn't look like there is very good VDI support for zfs. It seems to freeze when i try to write to a file system. I tried with the non snappy version too, just in case I mucked up something, but the vanilla version also froze. Now that I have a compatible version, I will test it on actual hardware, instead of in a VM. This build actually has snappy baked into zfsonlinux using the snappy-c library I found. If this route works, I will look at using the snappy-c port in an illumos build. I have all the changes out on bitbucket, and I did spend a few months learning the opensolaris/illumos build files....

I didnt expect to have the time, but I really became interested in answering the question of how easy working with the linux port of ZFS would be. I am here to say that I got everything compiled and installed last night. I am now trying to track down an shared library file that doesnt seem to be found. It exists, but my guess would be that the zfs tool is looking for it in /usr/lib instead of /usr/local/lib. The specific error is: /home/techsaint/files/zfs-project$ zpool zpool: error while loading shared libraries: libnvpair.so.1: cannot open shared object file: No such file or directory

My ZFS/Snappy project was put on hold, because of technical issues and lack of understanding (and documentation) of how to integrate a library into an Illumos build. I may have found a compromise. The zfs on linux project has become a bit more mature. I may switch up and use linux to create a snappy proof of concept.   I understand linux build structure much more than open solaris build structure, and the things i dont understand are fairly well documented. Because zfsonlinux is on github , I'll most likely be using Github and forking the project.

Woo!. Seems everything compiles perfectly. I had to copy the snappy libraries into the closed binary section of the build. Also, i had to build a 64bit and 32bit version of snappy (a bit difficult). I ran into a libtool bug (yes, another road block) when trying to compile the 64bit version of snappy. Right now, I use iscsi with virtualbox. So its difficult to import that into a different VM product (most desktop VM products dont support iSCSI booting).. I think i will copy over my illumos-gate directory to a now OpenIndiana installation, using a virtualHD, instead of iscsi. Then i should be able to import it into VMware or VirtualPC...and avoid the virtualbox bug I am experiencing with the latest illumos build. https://bitbucket.org/techsaint/zfssnappy

I am currently trying to add the snappy compression into ZFS for fun. You can check out my code here:  https://bitbucket.org/techsaint/zfssnappy I've been working on this for a few months (started around august). I've had massive roadblocks every step of the way. The first was the oracle purchase of Sun. Because of the purchase, the Sun Compiler tool patches (which were needed for an illumos build) were removed from the public. It took a month or so for the community to post instructions on using GCC instead of the Sun Compiler. After that, there was slight bug in the illumos build configuration that didnt allow for working builds if your new build version was older than the whatever version of illumos you were on. See  https://www.illumos.org/issues/1118 Now this new bug, is a virtualbox bug, which just appeared out of no where. Here is the ticket:  https://www.virtualbox.org/ticket/9651 . Since I rebased with the latest illumos code (as per build instructions) I ...

Just an update. I've been down in the "trenches", poking around with various technologies. Below is a short list of things i am checking out: Mono and SWT I am checking out mono and SWT as a UI toolkit vs Windows.Forms, or GTK# or WxWidgets or some other toolkit. SWT is an interesting UI toolkit that uses native widgets. Originally from IBM and now a major part of the eclipse project. It seems that IKVM allows SWT to be run without much effort. I'll post some of my progress on this later IKVM and Eclipse I am also checking out running the current version of eclipse with IKVM. Why? Its an interesting test on how far the project has come, its also could provide an interesting alternative to MonoDevelop or Visual Studio. WindowBuilder is also an interesting feature i'd like to check out. What if WindowBuilder could produce C# compatible SWT code? Could be a boon for Desktop mono linux apps. Solaris ZFS development I am looking at doing some Solaris Develop...