Recently I was trying to present a way for individuals to find patterns in large amounts of data. The end result was considered successful and I thought I would share at a high level what was done.

Recently I was trying to present a way for individuals to find patterns in large amounts of data. The end result was considered successful and I thought I would share at a high level what was done.
So what does that mean? Euler’s formula demonstrates, albeit for still not terribly clearly known reasons, that there is an intrinsic connection between complex exponential functions and trigonometric transformations.
In case you were wondering. No, it did not help solve the original business need. But it did help to continue to keep alive the awareness that everyday people I work with produce clever, and sometimes stunning, solutions with logical blocks. It’s like working with artists who craft in ether.
Sencha (previously known as Ext JS) has released Sencha Touch, a HTML5 Mobile Application Framework. which allows you to develop web applications that look and feel native on Apple iOS and Google Android touchscreen devices.
It makes use of HTML5 for delivering audio/video or localStorage and CSS3 for maximum styling like rounded corners, background gradients, and shadows.
The code created is resolution independent. It uses a method which allows developers to change the overall scale of their interfaces on the fly with no pixellation.
Sencha Touch has a powerful animation system that makes flexible animations between screens and views possible.
Slide, pop, and fade animations are included with the library, each with a robust set of options to change attributes like direction and masking style.
And, as they are created with CSS, building custom animations is a joy.
Also, it includes a set of common icons for using them in toolbars and tab bars.
Compatibility: Apple iOS and Google Android
Website: http://www.sencha.com/products/touch/
Demo: http://www.sencha.com/products/touch/demos.php
Search Engine Optimization (SEO) Toolkit is a free tool from Microsoft forimproving a website’s relevance in search results.
Features:
SEO Toolkit can be installed easily using the Microsoft Web Platform.
There’s a great post on it here: http://weblogs.asp.net/scottgu/archive/2009/12/15/search-engine-optimization-seo-toolkit.aspx
Consulting is difficult. You have to know a couple of things to get it all to work properly.
The last one can be the toughest… Typically you get to this juncture from one of three paths
Seeing those points written down starts to make it all seem easy but real life really does make the water much more murky. It is no coincidence that each sentence starts with “You have to know“. It is also no coincidence that these points apply equally to real life…
Success is measured as equally by inward as outward focus.
So, it is not easy but when any, or all, of these stars start to align it is best to bring the issue to front, acknowledge the possible mismatch, and if appropriate refer them to somebody else.
struct foo {
Very often there is a moment in a project where one or more people go “uh oh.” What happens next usually defines the final result. Typically it is one arm of the overly dreaded triangle that is the immediate root cause of concern.
It is important to note that quality is not an input. It is essentially the output of the sum of the three. The quality level you’re shooting for is not absolute — it’s really part of scope. You have to articulate and repeatedly verify the bar needed to satisfy your customers. If a system performs the functions you say you wanted and you still don’t like it, then you got the requirements wrong. If you update the requirements to address your objections, you’ll discover that the scope is greater than you identified. That confusion is a telltale sign of a team with immature design capability. (It should never be forgotten that Project management is a subset of Program management.)
Enterprise deployments of environments such as SharePoint require careful pre-planning, expertise, and thoughtful consideration for “future proofing” the final deliverable. A little now, or more later really does apply. Knowing the difference between scaling up or out is not sufficient. Sometimes you need to step back and redesign and redeploy everything along with updating the project initiating processes as well. SharePoint is a nebulous product which requires understanding of SQL, WSS, HIGs, peple, and more. 2010 is going to make things even more “difficult” so take your time, talk to everybody involved, and have a process.
When new technology is released it can be very painful. In more ways than one…
It can be painful to use if it is still in a Beta like form. Bugs, lack of features, changing road-maps, and other nuances of maturing code can be irritating for end users and those who support the solution. And with Google taking it upon themselves to redefine the public’s perception of “Beta” the waters are more clouded than ever as to what to expect from Beta software.
Rabid supporters and detractors serve to polarize and create opinions/sides that apparently must be held at all cost. Microsoft/Windows Vs Apple/OS X, Google/Android Vs Apple/iPhone, etc. Having an opinion is great, but if it blinds you then maybe it’s not so great…
New software frequently becomes the “in” thing. But just because it’s new and shiny does not mean that you should use it. Just because you feel that you need to learn it does not mean that it needs to be jammed into your current project(s.)
SharePoint 2010 has some rather interesting social networking features that are fraught with hazards without sufficient governance. Take it slow and really consider the consequences of feature deployment.
Monte Carlo Simulation is a technique used to estimate the likely range of outcomes outputted by a complex process by simulating the actual process with randomly selected data generating conditions that are true to the process model a large number of times. (In fact, the more you do it the better your data.) The Monte Carlo method is best applied whenever a deterministic solution would either be too computationally intensive or if such a solution does not exist whatsoever.
Monte Carlo Simulation is used in/with
Monte Carlo Simulation is not a “what if” process. What if’s require single point estimates and use deterministic modeling. Basically you are using best case, worst case, and so on. By using Monte Carlo you consume large random samplings, sourced from probability distribution functions, to produce a large range of outputs which in turn can allow you with greater confidence to produce a narrower range of outputs. In other words you are not using equal weights for each scenario.
Why is this pertinent? Well, stay with me on this one, Markov chain methods are extremely useful for generating sequences of random numbers to accurately reflect rather complicated desired probability distributions, via a process called Markov chain Monte Carlo methods. A tool that is used to generate simulations from a probability distribution…
The Google PageRank of a webpage is defined by a Markov chain.
And the penny drops…
Now, back to the point.
Depending on the degree of accuracy ultimately required, millions or billions of points may need to be tried. Distributing billions of point calculations across multiple servers running Monte Carlo Simulations via web services would parallelize the process and generates results VERY quickly. Good in concept but how to do it?
As defined by the W3C a web service is “a software system designed to support interoperable machine-to-machine interaction over a network.” Running web services on IIS has advantages not limited to:
Needless to say, unless your requirements can be served by parallel computations, which would have no dependency on others in the pipe, this is going to become very difficult or rather “challenging” 🙂 very, very quickly.
So how could SharePoint fit in? SharePoint is perfect for acting as a landing point for your data. In and out. Companies benefit by building intelligence into their document libraries and lists with workflows. With workflow, SharePoint can act as a central hub for the data, sending it out to a queue which distributes to nodes on the network. Upon return, the data could be used to populate lists, document libraries, notify people/groups, and more. Search, BDC, Security, and all the other features in SharePoint make this concept a compelling one.
I recently was talking with somebody who felt that TrueCrypt hidden volumes were the bee knees. The scenario they used, and which I myself have read ‘musings’ about, involved a laptop carrying sensitive corporate data being seized by customs. Laptop drive gets “reviewed”, secret container is not seen, and laptop passes as normal and uninteresting. Big deal. Bigger deal is if you have 007 style data and that guy in the uniform is pretty certain you have it as well. My colleagues version of the story ends with an almost hollywood style style exhalation of breath and cinematic zoom out to the hero walking out the door. That’s not how it would probably pan out…
Truecrypt volumes, which are essentially files, have certain characteristics that allow programs such as TCHunt to detect them with a high *probability*. The most significant, in mathematical terms, is that their modulo division by 512 is 0. Now it is certainly true that TrueCrypt volumes do not contain known file headers and that their content is indistinguishable from random, so it is difficult to definitively prove that certain files are TrueCrypt volumes. However their very presence can demonstrate and provide reasonable suspicion they contain encrypted data.
The actual math behind this is interesting. TrueCrypt volume files have file sizes that are evenly divisible by 512 and their content passes chi-square randomness tests. A chi-square test is any statistical hypothesis test in which the sampling distribution of the test statistic is a chi-square distribution* when the null hypothesis is true, or any in which this is asymptotically true. Specifically meaning that the sampling distribution (if the null hypothesis is true) can be made to approximate a chi-square distribution as closely as desired by making the sample size large enough.
So what does this all mean? Really nothing for us normal people. For those whom I have built custom STSADM containers for securing your backups and exports, your data is still secure and will stay that way indefinitely. For those running across the border. A forensic analysis will reveal the presence of encrypted data, TrueCrypt volumes or otherwise, but not much more. Sometimes that’s enough to start asking questions or poking further. With the forensic tools, not the dentistry kit.
* A skewed distribution whose shape depends on the number of degrees of freedom. As the number of degrees of freedom increases, the distribution becomes more symmetrical.