ZFS project

I recently began a storage project at home. Basically I intend to build a central NAS based on FreeNAS formatted with ZFS.

If you’re not aware of it, and shame on you if you indeed are not, under the Common Development and Distribution License (CDDL) in 2004, Sun released the Zettabyte File System as a means to bring advanced features like filesystem/volume management integration, snapshots and automated repair to its storage systems to its platforms. Since then, it has been fully integrated into OpenSolaris and Solaris 10, FreeBSD 7, and others. (Though I would steer clear of anything FUSE related for now…)

The challenge that I have been facing is how to get performance levels that are supposedly possible out of it. I have done the math on my hardware and know my goals. Getting them should be “fun.”

So far these links have been helpful.

http://wiki.freebsd.org/ZFS
http://www.freebsdnews.net/2009/01/21/setting-freebsd-zfs-only-system/
http://blogs.freebsdish.org/lulf/2008/12/16/setting-up-a-zfs-only-system/

Prime time for SharePoint? With Gazelle perhaps…

Google announcements are all the rage lately. The latest being the announcement of an impending Chrome OS. Exciting? Perhaps. Revealing? Definitely.

What really caught my attention was the removal of the beta tag from its apps on the same day.

It’s no coincidence. It’s prime time for this kind of web based OS offering and Google knows it. But if it is web based what happens when it is offline you ask? You can already use Gmail offline. Which in itself should be indicative of other app experiences in a totally web-oriented Chrome OS powered/supported with Google Gears. The same goes for Google Docs in offline mode, an option some people have been using for over a year. It also seems like an opportune time for the long rumored GDrive online storage to finally rear its head, picking up on the line “people want their data to be accessible to them wherever they are and not have to worry about losing their computer or forgetting to back up files.” That alone could make any Chrome based OS wildly more compelling.

So where exactly does that leave Microsoft? Doomed? Certaily most of the articles I read last night seemed to conclude with “Microsoft should be worried this time next year” or worse statements. I honestly do not think so. A new OS, based on a popular browser, could, and probably will, be a contender to say the least… Which is a good thing. Most markets need competition to stay healthy. Back in February 2009 Microsoft Research released an interesting paper [PDF] about a Web browser it calls Gazelle that’s constructed in such a way to act like an operating system with the browser kernel exclusively protecting resources and sharing across Web sites. Performance at that time was a definite issue, but the key point is that somebody somewhere in Microsoft saw this need and allocated resources towards it. With the new version of SharePoint supporting more browsers than ever I think the future actually looks rather rosy for Microsoft, thanks to Google.

Hohm is where the heart is…

If you’re green you will probably be interested in Microsoft’s Hohm. It lets people with participating energy providers track their energy usage online via a website. The end goal being to have the site give you tips on how to save energy and thus money.

The first participants are Pugent Sound Energy, Sacramento Municipal Utility District, Seattle City Light and Xcel energy. Not a huge amount of partners, but it’s a start. You can ask for access to the project by going to the page and following the instructions… Hohm

SharePoint Disaster Recovery: A moment

Disk space is cheap. We all hear and see it but plenty of you out there seem to ignore this fact. Yes, there can be a cost associated with maintaining the extra volumes in your data plan, but does there rally have to be?

Let’s face it, the average hard disk has a stated MTBF that is just ridiculous. Oft misinterpreted, and more generally misunderstood the numbers range upward of 50+ years. They are sourced roughly with the following logic. If a drive has a MTBF rating for 300,000 hours and the service life is 5 years a group of these drives should provide 300,000 hours of service before one fails. Needless to say, the unknown unknowns can interfere… The key point here is that they as a standalone device are supposed to be, and typically are, rock solid and reliable. Paired with a drive of equal properties from a different manufacturer, or if the same, from a different production batch, your odds of failure are even more reduced. Right now an external 1TB drive with USB or Firewire will run you less than $150. Buy two and you’re still under $300. Total costs for electricity ~$50 a year? That’s cheap.

Now why don’t people just hook one of these to a server, networked would be a bonuus, and add it in as an additional backup location? Some do, but they are the exception, not the norm. More than once, though sometimes it took some “cajoling”, clients of mine have seen the merits of extra, cheap, storage that STSADM can dump data securely onto and be retrieved quickly and easily. I’m a firm believer in the more baskets you have, the fewer broken eggs you have.

Needless to say you can secure these drives with something like this…

Opera Unite – a perspective change from the centralized model used by SharePoint?

Opera Unite, a web browser melded with a web server. Now there’s a novel concept.

Opera Unite allows you to share your files, stream music, host sites, and communicate real time with people. The suite of services, that’s what they literally are, are comprehensive.

  • File Sharing
  • Photo Sharing
  • The Lounge
  • Fridge
  • Media Player
  • Web Server
  • and more…

But there’s a problem with it. A very big problem that I suspect Opera Marketing are all too aware of. Although Opera Unite claims to “directly link people’s personal computers together,” to use it you must have an account on Opera’s servers. Once you have that all of your exchanges pass through Opera’s servers first. Sure, that’s an effective way to get around technical difficulties such as NAT, firewalls, etc, but the big issue is that it makes Opera the intermediary in your social interactions — not Facebook, not MySpace, but Opera. Think it through. Stepping past all the hype, the benchmarks*, etc. you have just another lockin scenario. Opera is up you’re up. Sure your stuff is on your machine but it can only be accessed via Opera the domain.

Is there a way around this? Do we need a way around this? Yes, it would be possible to create a swarm and find your friends, but what happens when your computer is down and somebody wants to access your content. Nothing.

*Benchmarks

Excerpt from http://unitehowto.com/Performance below. Take them in context.

Opera Unite uses very smart file I/O! Even if you save data to file each request (simplest, but stupidest way to do it) – it still can push out very impressive 744 requests/second! (It probably means that this data is saved to memory and dumped only sometimes, smart move!)

It seems like Opera uses 13 threads (seems like a soft limit, but unchangeable). 13 concurrent connections max out @ 810req/s, 1.23ms processing time.

For comparison:

PHP+Apache(+MySQL) is almost 2 times faster than peak Unite performance.

Compiled C++ web server (MadFish WebToolkit ) is only 6 times faster than Opera Unite, but that is compiled raw C++.

nginx (one of the fastest Web Servers available) is only 5 times faster than Opera Unite (clocked at 4900 req/s in raw C++) “Welcome to nginx” cycle (no I/O or scripting).

iPhone OS 3.0


Looks like the field will level. And about time…

Two cool additions to the iPhone that now use HTML5 features:

Safari JavaScript Database Programming

The HTML 5 specification provides a new mechanism for client-side data storage: JavaScript database support. HTML 5 is currently in development by the Web Hypertext Application Technology Working Group (WHATWG).

JavaScript database support is available in Safari 3.1 and later, and in iPhone OS 2.0 and later.

You should read this documentation if you are a web developer who wants to store data locally on a user’s computer in amounts beyond what can reasonably be stored in an HTTP cookie.

HTML5 Application Cache

Michael Nordman of Google asked “Is this stuff built into shipping Safari or iPhone browsers yet?”

David Kilzer replied “This feature shipped with iPhone OS 2.1. When you use “Add to Home Screen” from the “+” button on Safari for iPhone, a web application with a manifest defined (per the HTML5 spec) will be saved with any cached resources. Note that the manifest file *must* be served with the correct MIME type for this to work.

There is no shipping version of Safari for Mac OS X or Windows that supports this feature yet.”

dd: clean your drive securely

Now like anybody I’m a BIG fan of wiping old drives using dd but sometimes there’s a tool out there that will do most if not all of the work for you. Cue DBAN. OR as the site says:

Darik’s Boot and Nuke (“DBAN”) is a self-contained boot disk that securely wipes the hard disks of most computers. DBAN will automatically and completely delete the contents of any hard disk that it can detect, which makes it an appropriate utility for bulk or emergency data destruction.

Complemented with TrueCrypt you will have a mighty secure setup. Possible / definite paranoia issues too… But your data will be secure. For the more command line orientated the old reliable dd if=/dev/urandom of=/dev/disk bs=1k is good enough imho. (It puts random bits in place as opposed to a regular pattern. Not that it will stand up to NSA level scrutiny but it’s more than enough for most data recovery…)

For more go to:

DBAN: http://www.dban.org/
TrueCrypt: http://www.truecrypt.org/

Windows 7: Aka Watch Microsoft Rinse and Repeat the same mistakes

So… the detail is out and it is not impressive for a variety of reasons. If any of you recall a screenshot of a literal plethora of different versions of Windows 7 floating around a while back, you may now smirk and / or groan. Yes, Microsoft have, in my opinion, once again shot themselves in the foot.

The following will be the actual new SKUs for the OS:

  • Windows 7 Starter (limited to three apps concurrently)
  • Windows 7 Home Basic (for emerging markets)
  • Windows 7 Home Premium (adds Aero, Touch, Media Center)
  • Windows 7 Professional (Remote Desktop host, Mobility Center, Presentation mode)
  • Windows 7 Enterprise (volume license only, boot from virtual drive, BitLocker)
  • Windows 7 Ultimate (limited availability, includes everything)

This is verging on idiotic, there is no sane need for so many versions of Windows 7. All that is needed is:

  • Minimum
  • Home Media
  • Enterprise

This is naught short of a cash grab and is not going breed any goodwill. Lead the way or make way still applies. Feel free to call me Microsoft… Or just get out there and read /.

Nifty: Microsoft Business Data Catalog Definition Editor for Microsoft Office SharePoint Server 2007

Ah Microsoft, you giveth and you taketh.


The Microsoft Business Data Catalog Definition Editor for Microsoft Office SharePoint Server 2007 aka MSBDCDEFMOSS2007 (all me…) is a nifty yet hobbled tool fresh(ish) from Microsoft. In brief its features include:

  • Underlying XML is abstracted by the design surface and properties window
  • Drag and drop web methods, tables, or views to create line of business (LOB) connections.
  • Entities and methods are created automatically from database metadata and WSDLs.
  • Additional method instances can be added to further enhance the database or web service connection.
  • Method instances can be tested from within the tool, enabling incremental development of LOB connections

That said… Here’s the catch. (For now I hope…)
The tool is unusable with SQL Server 2005/2008 schemas… So, you cannot use it against AdventureWorks, BUT you can use it against AdventureWorksDW. If you dare to try, you will be rewarded with a “Could not process Table ‘….’. Also, make sure you have SELECT Rights on the Table/VIEW”.

Helpful links