It's not art, it's design

I recently read Eric Ries' Lean Startup, it's excellent, a must read for anyone doing any sort of product development, not only software.

The nutshell version that I wish every developer & designer should would take to heart:

What you're building is not art, it's a functional workflow. Your design is an experiment to make that interaction the most satisfying user experience possible. You are not your user.

New beginnings, new ideas

What an amazing time to be a developer. It seems like 10 year old hypothetical situations have suddenly exploded into reality. Node.js, github, NoSQL, Metro, WinRT, ASP.NET OSS, it's like a whirlwind. As someone who's always specialized in not specializing it's almost overwhelming to try keep up.

What I do know is this:

The web won. 

So long Silverlight, Java and Flash. JavaScript wins (as frightening as that sounds) I think we owe "alert" a bit of respect, time to learn how to do it properly, even on the server.

Javascript, HTML & css are now the lingua franca of everything, not just the net. If you think Apps are better, you miss the point, you can write apps in HTML and Javascript thanks to PhoneGap and Metro. (I can't come up with a decent acronym for it, JHaCss? CJaH? CEH? - techincally it's ECMAScript).

Even databases, I don't think it's a case of "Why NoSQL?" anymore. In most use cases I have to ask "Why SQL?", and do I really need to host my own? (Quick plug for Cloudant and MongoLab ;) ) 

Javascript still sucks.

As my Mum always used to say, winning isn't everything. And my the rules of  portability don't change. Write once, debug everywhere holds more than ever. If you're fluent in something like C#, javaScript is incredibly aggravating. 

To the point,

I'm starting a new learning app, it's a Metro C# music sequencer I've dubbed "Seque Beat" for a bit of fun.

https://github.com/doobiwan/SequeBeat

Nothing really there at the moment, I want to explore portable assemblies, TDD, all the good stuff, feel free to join in.

Yay for Metro and Windows 8! Boo for lazy haters .... ;)

Why do all my posts start as comment posts.....


Look, consider me a bit harsh if you will, but if a developer considers themselves just a "Silverlight Developer" they deserve to feel pain. No one technology can solve every problem, the beauty of ".NET", or lets rather say "Managed" applications if you will, is not write once run everywhere, that was debunked a good 10 years ago after Java failed miserably at it. What C#/XAML/Visual Studio/LINQ give you is a set of very powerful, specialized tools you take with you and use as necessary.

I do windows I do web, I do services, I do some Silverlight, and I'd be happy to tackle Mac, Android or iOS with Mono if the opportunity ever presented itself.  What ports is the language and the patterns, the rest is too difficult to abstract over such diverse platforms. I learnt that first from Miguel de Icaza (who weighs in fantastically here http://tirania.org/blog/archive/2011/Sep-15.html) and secondly from  painful experience.

And that is exactly the issue with Metro.

Under the hood Silverlight is the closest cousin to WinRT there is, they're both subset CLRs based on native implementations rather than pure CLR due to constraints - Silverlight needs to be tiny to be downloadable, WinRT needs to be as performant as possible. Silverlight is designed for browser downloadable RAD desktop apps. WinRT is designed for App Store sourced, "run on any processor" Touch apps.

I won't pretend to be fluent in architect, but I guess you could call them domain specific language implementations.

So now what?
If you really want reuse what you should really be looking at is WCF and/or REST. Not necessarily even server side, run a windows services and make your Metro, Silverlight, HTML, WPF and, well frankly anything, just a thin UI implementation.

Just remember, every time you over engineer, a Project Manager kills a kitten.

On Frameworks

This was a comment in response to a post by Jeremy Miller. It got a bit wordy so I figured I might as well keep it ;) 


I love the idea of choice, I love that there are multiple Frameworks, and I would love them all to be uber successful.

But... It comes down to portability, long term support,  company culture, and having a common toolbox. 

I've been at this game for over a decade, and the biggest problem I  encounter over and over again (besides beancounters ;) ) is legacy frameworks. By that I mean things that in their day were considered by someone as the tool de jour, but have fallen out of favour and/or and maintenance and now hold products back from forward migration, or keep the people maintaining those apps from progressing their skills. Tools that effectively require developers to become specialist of themselves.

It could be anything  - COM components, CMS frameworks, web Frameworks, DataSets, XML islands, Microsoft Entlib ... ;)  etc etc

Developer "A" implements "Excellent Architecture Foo" and alls good, because he understands and buys into it. He leaves, developers "B" comes along that has a different architecture perspective and it doesn't gel. Something like a culture change from "Software Factories" to YAGNI. Everything turns into a hack, and next thing you have is spaghetti (usually with meatballs). 

The way I (try to) avoid that is by very careful consideration. Choosing between Fubu, OpenRasta, MonoRail and ASP.NET, is not just about their relative merits, but their long term impact on my products and my teams, and our commitment to maintaining those products. It means every new hire has to be upskilled, and in many cases it's not the easy stuff you need skills in, like building pages, it's the hard stuff, like debugging and performance which only comes with experience.

That's why developers migrate towards the "most popular" frameworks, it's self preservation. In Java it's Spring, in Ruby it's Rails, in .NET, well, it's "Microsoft sourced" because it's the usually the largest support team. It's not always the case. nHibernate, JSON.Net, MVVM Light, log4net, DotNetZip come to mind as OSS tools that have managed to build enough momentum, and stay portable and relevant. 

Introducing “Sydney Trains & Ferries”

So my first app submission, it’s a great feeling.

There’s a little bit more the ST&F than a phone app with a bit of data so I thought it’d be a good idea to break it all down. ST&F is probably more Azure than it is Windows phone. Not expecting to make much on it also presents some interesting dilemmas and really challenged me to be razor sharp about what’s really important when it comes to cloud architecture. In other words – cost.

The WP7 app

The app is relatively straight forward. While uniqueness can be good, sticking to conventions allows users to focus on achieving their goals rather than wowing at chrome.

I need to give a big shout out to the developer community, for tools and guidance:

• http://sterling.codeplex.com/
• http://mvvmlight.codeplex.com/
• http://qdfeed.codeplex.com/
• http://c4fdevkit.codeplex.com/

The source data

ST&F gets its data from http://www.131500.com.au. Once unzipped, it’s a 1.5-2GB XML dataset comprising about 70 different transport services in and around Sydney. The schema itself is quite challenging to deal with. My first attempt at transforming a complete dataset took around 3 hours to run on my desktop PC, which is far from ideal. So as they say … to the cloud!

The data upload routine has now been farmed out to windows azure worker roles, the transformation now takes around 15 minutes to run using 4 worker roles, spinning up the instances often takes more time than the run itself (hopefully an issue that will be addressed by the azure compute team one day). So for about a dollar I save myself a whole bunch on frustration. It still has it’s issues – some documents are co-dependant, so there’s some workflow involved that’s invoked manually. In the near future this will be mitigated, but that’s a subject for another post.

Oh, did I mention I despise XML namespaces? They are everything that’s wrong with the world …. well almost.

The eternal problem of (current) cloud services

I’ll give you 2 guesses …. oh you got it on the first – cost.

While excelling in being able to handle the cost of burst loads like data transformations effectively, what cloud compute sucks at is low volume. Minimum cost of an SLA compliant web solution is US$180. Eeep. Sure you can use a stand alone extra small instance, but you’re still looking at $40+, and for less that that I can use old fashion web hosting, where’s the fun in that?

Time to rethink what you thought about ‘web services’.

App Data

What I realised while building the app is that while on the surface one thinks of “querying” service information, in fact, most lookups are unique key based requests like “Get Service Times For Stop Location”. As my previous posts spoil – I started from a typical “Table/Entity” mindset and tried this in Azure table store with frustrating results. It’s a bit challenging querying Table Store from WP7, which meant I’d most likely have to run a web role, which was a cost I was keen to avoid.

Taking a step back I realized that being able to identify each request is actually unique – things like “Chatswood Station” or “Bankstown Line” means each request can be represented in REST, so, blobs to the rescue! The data processing done in azure now transforms hierarchical semi relational data into thousands of static blobs. Sure there’s a ton of duplicated data, but blob store is cheap as chips per GB and I don’t need to run a web role. I have to watch out for the number of requests, but that’s mitigated by caching in the Sterling database.

In Summary

I had a lot of fun, but this is just the beginning. First the phone app needs to pass cert, then there are still a number of perspectives and views I need to present. The azure processing pipeline is also undergoing some re-engineering, so stay posted.

Here we go …

Sydney Trains & Ferries (beta) has just been submitted to the Windows Phone 7 AppHub. Here’s hoping everything goes smoothly.

As they say “Necessity is the mother of invention”, and in this case I needed an app for my own daily commute, hence Sydney Trains & Ferries was born. Admittedly the app is far from ‘complete’, the intention is to evolve through user input, so hop onto the feedback link and get screaming at me

The first think I have to ask is:

Do you want better Rail and Ferry info first or should I focus on getting bus info into the app?

Other than that, I’ll put together a few technical posts on what’s going on under the hood. Sydney Trains and Ferries is running on Windows Azure and presented some interesting problems, some of which I’m still busy solving.

Stay tuned.

The future of cloud computing

 

I like to think I get cloud computing. I like to think I get it pretty well from a philosophical perspective as well as a commercial one. With the mass commercialization of cloud computing underway, the message seems to have been distilled down to “rent your pre-existing configurations on the web, oh and we’ll come up with a scalable store”. Rackspace, Amazon and Microsoft all do exactly that  and they’re selling the vision short.

These ‘first generation’ Cloud services remind me a lot of the anecdote of the first cars. They looked like motorized carriages because that was the prevailing point of reference, no one took the time to re-evaluate the actual problem the carriage solved, removed horse specific requirements and built the car around the engine, not the other way around. I think that’s where cloud computing is currently “Carriage cars”.

There are those that get it. I think Heroku gets it, and subsequently AppHarbor. Funnily enough I think GoDaddy gets it.

Current large scale cloud hosts maintain a billing granularity of one “machine”. They infer that an application requires a machine to run, and subsequently 2 machines for failover. Under certain circumstances that may be true, but for most applications, especially new startups, activity could largely be measured in single digit percentages cpu time, basically putting 2 whole VMs to waste.

In other words, VMs are not a unit of work.

Even these services offering of PaaS fall short of what I consider ideal. They’re basically automated or programmatic ways of provisioning services that would otherwise require IT staff. In other words, it doesn’t abstract away the problem it just turns an IT problem into a programming one.

In the other corner we have Heroku and AppHarbor, and certainly a few I don’t know. Abstract away the hardware completely, there’s no waiting 15 minutes to “spin up a new server”. It works because apps don’t really need VMs. VMs are an easy out for the providers, they’re not good solutions for developers.

Funnily enough that brings me to GoDaddy of all companies. I recently saw they’re offering “4GH” hosting. Basically it’s auto scaling clustered web hosting services. They offer it at entry level prices closer to traditional web sites and my immediate reaction was “exactly”. With IIS isolation and throttling I don’t need a whole VM to run an app. I don’t need two servers for redundancy, the point of clusters is to distribute load. Take a few hundred applications and span them across a dozen servers, that’ll give 90% of users what they want at a fraction of what “cloud” hosting currently costs.

Goodbye tables, hello blobs

Okay, so after working with azure table storage for two weeks I can finally say I get it. Or more appropriately, I get what it’s not. It’s not, “table” storage …

If you think of a table of data as a series of rows of related entities which you can group and query you can be easily misguided into thinking you get azure table store. In reality Azure table store is more like “Azure Indexes” because in order to make them useful you have to contort you data for each view you wish.

For example, if you want to "group by” you’re better off creating a new table by pivoting your data on the group term. If it’s  particular record you want, you could search on partitionkey and rowkey, BUT, you could just as easily make that a single row partition, or … a blob.

That’s the situation in my use case. I thought because I am reading semi relational data a table would be most appropriate. But I’ve come to the conclusion that actually, I’m better of sticking JSON objects in blob store.

Oh, Table Storage ...

I'm having a ton of fun with Azure, but the one "version 1.x" feature that bugs me is the immaturity of table storage.

The fundamental ideas behind table storage are great - hi speed, low cost, non relational data. The issues I have are with the details of the implementation.

First off there's the supported structures. Why to we have to use flat objects? Why limit things to, basically,  a property dictionary, when  the semantics support graph serialization? Maybe constraints are good, but I find this limits the transparency of working with Table storage.

My next issue, while it's all well and good using 'similar' semantics to other technologies like Data Services, LINQ and Entity Framework, it kinda defeats the DRY principle in that it's not the same. Why was table storage not provided as a provider for Entity framework, as well as an abstraction over Data Services?

Lastly that brings me to Dev storage. Look, having local easy to configure tools is great, but there are known issues that just hurt the experience Oliver Jones covers some details here: http://deeperdesign.wordpress.com/2010/03/10/azure-table-storage-what-a-pain-in-the-ass/

My thought is, why mess around "faking" table storage over express anyway, when the whole point is to avoid SQL Server?

I figure the semantic and dev fabric issues could actually be solved quite easily, together - release a 'production' ready local version of table storage. This would help unified architecture and the dev experience.

Windows Azure Table Storage stupid mistakes

Don't program when you're tired ...

I've been working through the Azure labs and as usual I don't stick exactly to the script (how're you supposed to learn anything if you don't challenge the boundaries? ;) ) So after 2 late hours I couldn't figure out why:
"storageAccount.CreateCloudTableClient().CreateTableIfNotExist("ThingOnes")
"
Created tables, but the more common
"CloudTableClient.CreateTablesFromModel(typeof(ThingContext),

                                                    storageAccount.TableEndpoint.AbsoluteUri,

                                                    storageAccount.Credentials);"

didn't.

I just noticed now that in my "ThingContext" here:
"
        public IQueryable<ThingOne> ThingOneEntities

        {
            get
            {
                return this.CreateQuery<ThingOne>("ThingOnes");
            }

        }
"
the property name of the entity set was different to the EntitySetName I declared.

Setting them the same made everything happy again.

In other words ... magic strings are evil!

Windows phone 7 impressions : 2 weeks on

So it's 2 weeks on since I finally got my grubby mits on my Omnia 7, and overall I'm pretty happy with it. It's solid without being cumbersome and the UI is really slick.
The AMOLED screen is a marvel of crispness.

Having said that the phone, as any v1 product, has issues. The most jarring being without doubt the combination of tombstoning and capacitive face buttons. Being unceremoniously dumped out of "the harvest " and having to wait a minute again to resuming is something that would prevent me recommending the phone.

Overall though, I'm pretty excited about what should be in the pipeline for wp7 when I see top class apps like ninemsn and bye bye brains.

That and the fact I managed to type this entire post on my phone without issue is a testimony to both the keyboard and the browser.

Oh and the Omnia 7 camera is pretty good, I just don't get the reasoning of having  the default video format unplayable on the Xbox 360.

From a dev perspective the tools are really excellent, with a bit commitment I should have something presentable soon.

Silverlight ... HTML5 ... *sigh*

*sigh*

Steve Ballmer and Bob Muglias comments on HTML5 & Silverlight are nothing new. Well, except it being by the CEO, the relevant VP and at PDC. I guess that can look worrying if you want to it to.

I've actually found the fallout quite positive. In truth their comments were completely expected, if Microsoft was trying to "take over teh world" by installing silverlight on every device I'd say they were foolish and naive. It failed for Java, it failed for Flash, why would Silverlight be any different? Silverlight fulfills a different role to html, that explaination was the only bit missing from the keynote.

That's what's been great about the aftermath: The role and future of Silverlight has never been better expressed in so many ways and in such detail. If these perspectives had been made clear during or before PDC it would have been a none issue. Here's some of my favourites:

Bob Muglia's response
http://team.silverlight.net/announcement/pdc-and-silverlight/

Great detailed post from Mike Taulty
http://mtaulty.com/CommunityServer/blogs/mike_taultys_blog/archive/2010/11/01/silverlight-versus-html5-really.aspx

MVPs Jeremy Likeness & Laurent Bugnion also weighed in nicely 'from the coal face' on the subject:
http://csharperimage.jeremylikness.com/2010/10/so-whats-fuss-about-silverlight.html
http://csharperimage.jeremylikness.com/2010/11/silverlight-and-html5-microsofts.html
http://blog.galasoft.ch/archive/2010/10/30/my-position-on-the-silverlight-debate.aspx
Rather go look at the portable library project (http://player.microsoftpdc.com/Session/638f610a-ea51-4aef-9657-e3fe425ae745/469.163) to go see exactly what makes Silverlight so awesome.

Think of Silverlight as more “Thin Client” than “Rich Web”

There's a lot of justify your platform going on at the moment. I see Mike Taulty has weighed in on the subject.

The thing is that the 'HTML5 lobby group' aka Apple and Google, have a lot to gain by denigrating Silverlight and Flash. iOS doesn't support it and Google's core strength has been by commoditizing the web with its thousands of JavaScript programmers. HTML5 is all well and good, but in the rush to save the universe people are getting a bit blind. Add these two behemoths to the voices of the thousands of "script kiddies" that consider a couple of lines of copy and pasted PHP and JavaScript programming, and you'd think we're reached software Nirvana.

The truth is HTML is still HTML and Flash and Silverlight are not just about streaming video.

Even using a better browser like Chrome or Firefox, how many times have you been frustrated waiting for a page to load? Have you gotten irritated, or worse – had to apologize to a customer, while trying to find what you were doing in a mass of tabs or accidently closed the wrong tab and lost their state? What about sitting waiting for something to happen while JavaScript has a meltdown and there's no way of knowing? That's just from a user perspective. From a developers view, JavaScript is a complete PitA, and trying to do LoB or Verticals levels of complexity is a killer, never mind the testing time.

Then there's the whole question of running disconnected. Page storage is all well and good but what happens when *shudder* you have to move to another page? What about device integration?

The truth is Silverlight (and Flash) presents real value to application developers but is a bit difficult to quantify if you haven't felt the pain of trying to develop large HTML apps.

But that's just me.

Dear Microsoft, you were SPOT on. Not.

I guess it's the same old tail. Microsoft, a good idea, squandered. Right idea, wrong era.

...Picked up and owned by Apple.
The iWatch Is Real: It&#8217;s the New iPod Nano [PIC]

SPOT was Microsoft's attempt at smart watches, announced back in 2003, and after 5-6 years of spinning it's wheels it was canned. Barely 2 years later and Apple are back with a real how-to.

Music, Phones, touch PC's, Watches etc etc. Apple gets it right first time and executes in a focused manner. There's a lesson to be learned there.....

Why ‘web’ doesn’t ‘win’

If you haven't heard, apparently HTML5 is going to kill native apps. Awesome. Please tell my competitors that. Convince them in fact. I'll stick with my "obsolete" native apps, weather the hype and pick up their customers when they all decide to jump ship.

There's a great series of articles up on Wired at the moment, well worth the read for anyone pondering the future of the internet

http://www.wired.com/magazine/2010/08/ff_webrip/

Silverlight Deployment dispatcher

Note to self. I encountered this problem when trying to access WCF Data Services from a Silverlight App, specifically when running out of browser. Trying to set a collection that's bound to a UI thread throws a cross thread exception, you're supposed to use the UI Thread dispatcher.

A lot of examples suggest you use the Application layout root dispatcher, but Shawn Wildermuth (as is so often the case, thanks Shawn!) has the right answer. You need to use the Dispatcher in the Deployment class as such.

Deployment.Current.Dispatcher.BeginInvoke(
        () => results.Text = "Pooled Dispatcher Worked");

Works like a charm.

Windows Phone 7 Launch Games

Awesome. Apps? What Apps?
http://www.engadget.com/2010/08/16/xbox-live-launch-titles-for-windows-phone-7-finally-revealed-we/

Every time I see great demo's like that, my aspirations for world domination collapse ...

Snip of the week

Without a doubt the most interesting bit of news last week was Oracle suing Google over Java.

The best comments coming form Miguel Icaza of the Mono Project. Most notably:

Unlike the Java patent grant, the Microsoft Community Promise for both C#, the core class libraries and the VM only require that you have a full implementation. Supersetting is allowed.

Additionally, Microsoft has placed the .NET Micro Edition entirely under the Microsoft Public License which comes with an even more generous patent grant, and covers a superset of the code covered by ECMA/ISO 335.

We have open source implementations of both, and even more luckily, the ECMA/ISO VM specification allows for different profiles, to allow for ultra-small or server-sized versions of the VM to be created. Ideal for mobile platforms.

Google could settle current damages with Oracle, and switch to the better designed, more pleasant to use, and more open .NET platform.

 I really like the Mono project and hope this does them some good, especially with the success of MonoTouch and the upcoming release of MonoDroid. With cross platform skills becoming more of a necessity than a nicety, being able to present a unified C# story is a great prospect.

Hello Blogspot, long time no see ....

It's good to be back. After trying spaces for what seems like a painful eternity I'm happy to be back. the old stuff is still there, well at least until I figure out how to port them across.

http://dubiouscode.spaces.live.com/blog/