WPF for the Web guy…a story of pain, despair and learning

Scott Galloway — Wed, 20 Jan 2010 20:53:59 GMT

Well, OK I’m over dramatizing it a tad :) As you may have noticed from my last post I’ve started working of some stuff which is outside my comfort zone. I recently joined a company called cozwecan,working for a chap named @RobertTheGrey, the first project we’re working on is a photo sales site…you know kind of like Getty Images / Smugmug / Flickr except for direct photographer-customer sales. As the only full-time developer (so far!) I’ve been tasked with building the vast majority of the applications (be they web, desktop, etc…) which we need for the business to work.

Obviously one of the most critical bits for an e-commerce site is stuff to sell! In our case this is a stock of really high quality photographs taken by professional photographers around the world. First question; how do we get these images on the site, and how do we let the photographers add information to their photographs to let them be found by folks who want to buy them? Well, that’s the first problem I’ve been asked to solve!

When working on the requirements, it quickly became clear that this had to be a desktop app, it needed to allow operations on multi-megabyte photographs, needed to allow ‘tagging’ offline. Especially during initial loading, there’s 10-100s of thousands of images getting worked on…forcing this to be online adds a nasty lag and makes the task a pain in the butt to complete. Above all, it needed to be reliable!
Now, in my dev head this led to a number of decisions about the features the app would need to support:

1. Intuitive, responsive UI; the users are not necessarily computer-savvy, need to use obvious UI metaphors and make it fast enough to be pleasant to use.
2. Needs to do some offline image processing; we’re planning on hosting this on the cloud….CPU time costs cash :)
3. Should work offline
4. Needs to be built around reliable upload of multi-meg files (across potentially tiny connection bandwidths).

So, these are the basic needs…technically this leads me to some technology choices:

1. Multi-threading / concurrency (as with any real desktop app) is key…as is a decent Desktop technology…
2. See above, we need to ‘watch’ for new files arriving, make thumbnails transparently etc…
3. So, it needs some sort of persistence…along with other non-core app requirements (e.g. tagging) we’ll likely need a little, lightweight DB.
4. Needs ‘block’ based uploads with retry / verification of uploaded items…

Luckily I’ve been a dev for quite a while and had at least some idea where to start…the biggest challenge would be learning!
Firstly, the obvious desktop platform de jour is WPF…this is in essence the current replacement for WinForms, it’s VERY customizable but has a somewhat infamously steep learning curve. I’ve spent a bit of cash on books on WPF, followed all the PDC presentations, podcasts etc…on the subject matter. After 3 weeks I’m finally getting comfortable with it…
Second, I needed a DB…needed to be lightweight, needed to support LINQ (what, I like LINQ :)), needed to support some sort of ORM technology.
The most obvious choice was SQL Server Compact Edition, a kind of cut down SQL server which is great for desktop apps…however…I could not get it working! Turns out it’s really sensitive to the versions of various installed components. This was a red light for me…I needed the DB to be as stand-alone as possible! That left one obvious choice:
SQLite, from the website:

‘SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine. SQLite is the most widely deployed SQL database engine in the world. The source code for SQLite is in the public domain.’

Perfect! For .NET support, I used the SQLite ADO.NET provider (there is a C# only port of SQLite, but it’s slower and not needed for my scenario. Deployment is literally including a reference and creating a file…now it does have one pretty severe limitation which I’ll cover later :)

My ORM choices were nHibernate or EF…there being support for SQLite in both of these platforms.
Now, I have used nHibernate for a recent project (disclaimer, I left fairly early in the project :)) and one of the things which annoyed me was the large number of dependencies, and the seeming fragility across versions of these dependencies. For a low impact desktop app, this was just too large a footprint and too big a risk. So I had to count it out.
This left me with one choice…Entity Framework. Now, this wasn’t an easy choice…I have used EF in the past (tinkered with it) and found it a painful experience…(others have too which led to this). Now, I’m not getting into the politics behind MS choice to push EF (and yes, it was a political choice), but V1 is fairly buggy, and for me at least fairly unnatural. This led to my second learning ‘opportunity’…Master Entity Framework! As I write this, it’s literally 2 hours since I worked out my last bit of EF pain…which led to this model:

Looks simple…well, yeah…but it’s taken me AGES to get to this…I know SQL really well, C# REALLY well, but the mix of obscure concepts, poor documentation and obfuscated error messages makes working / learning EF way too hard! EF 4 (in VS 2010 / .NET 4) improves this…and it can’t come soon enough!

So, that’s worked out…I hope :)

So, I mentioned previously that SQLite has a bit of an issue…it doesn’t really like concurrency. Well, that’s an overstatement…it is a multi-read, single write system. Once you realize it’s an issue it’s fairly easy to solve. I use this pattern:

public static void AddRange(IEnumerable<UploadFile> files)
        {
            using (PixEntities ent = new PixEntities())
            {
                using (new ReadLock(entityLock))
                {

                    using (new WriteLock(entityLock))
                    {
                        foreach (var file in files)
                        {
                            ent.AddToUploadFiles(file);
                            ent.SaveChanges();
                        }
                    }
                }
            }
        }

As you seen, this has two main elements…I use a ReadLock (where I would normally read *from* the DB, in this case umm, I don’t :)) then I use a WriteLock to wrap any operation where I write to the DB. The observant will also notice what looks like a bug…I do ent.SaveChages() for each iteration of the loop…this is another SQLite quirk, it doesn’t support batch updates…and gives a weird error about file locks if you try :)

The little locking class I use is from here, but I’ve put it below for your use…anyway, this is part one of several parts of my experiences in working on cozwecan…stay tuned!

  public static class Locks
    {
        public static void GetReadLock(ReaderWriterLockSlim locks)
        {
            bool lockAcquired = false;
            while (!lockAcquired)
                lockAcquired = locks.TryEnterUpgradeableReadLock(1);
        }


        public static void GetReadOnlyLock(ReaderWriterLockSlim locks)
        {
            bool lockAcquired = false;
            while (!lockAcquired)
                lockAcquired = locks.TryEnterReadLock(1);
        }


        public static void GetWriteLock(ReaderWriterLockSlim locks)
        {
            bool lockAcquired = false;
            while (!lockAcquired)
                lockAcquired = locks.TryEnterWriteLock(1);
        }


        public static void ReleaseReadOnlyLock(ReaderWriterLockSlim locks)
        {
            if (locks.IsReadLockHeld)
                locks.ExitReadLock();
        }


        public static void ReleaseReadLock(ReaderWriterLockSlim locks)
        {
            if (locks.IsUpgradeableReadLockHeld)
                locks.ExitUpgradeableReadLock();
        }


        public static void ReleaseWriteLock(ReaderWriterLockSlim locks)
        {
            if (locks.IsWriteLockHeld)
                locks.ExitWriteLock();
        }


        public static void ReleaseLock(ReaderWriterLockSlim locks)
        {
            ReleaseWriteLock(locks);
            ReleaseReadLock(locks);
            ReleaseReadOnlyLock(locks);
        }


        public static ReaderWriterLockSlim GetLockInstance()
        {
            return GetLockInstance(LockRecursionPolicy.SupportsRecursion);
        }


        public static ReaderWriterLockSlim GetLockInstance(LockRecursionPolicy recursionPolicy)
        {
            return new ReaderWriterLockSlim(recursionPolicy);
        }
    }


    public abstract class BaseLock : IDisposable
    {
        protected ReaderWriterLockSlim _Locks;


        public BaseLock(ReaderWriterLockSlim locks)
        {
            _Locks = locks;
        }


        public abstract void Dispose();
    }


    public class ReadLock : BaseLock
    {
        public ReadLock(ReaderWriterLockSlim locks)
            : base(locks)
        {
            Locks.GetReadLock(this._Locks);
        }


        public override void Dispose()
        {
            Locks.ReleaseReadLock(this._Locks);
        }
    }


    public class ReadOnlyLock : BaseLock
    {
        public ReadOnlyLock(ReaderWriterLockSlim locks)
            : base(locks)
        {
            Locks.GetReadOnlyLock(this._Locks);
        }


        public override void Dispose()
        {
            Locks.ReleaseReadOnlyLock(this._Locks);
        }
    }


    public class WriteLock : BaseLock
    {
        public WriteLock(ReaderWriterLockSlim locks)
            : base(locks)
        {
            Locks.GetWriteLock(this._Locks);
        }


        public override void Dispose()
        {
            Locks.ReleaseWriteLock(this._Locks);
        }
    }

Threading update...what I actually do...

Blog Author — Tue, 25 May 2004 00:03:00 GMT

Should mention I suppose the I lied in yesterdays post on threading using .NET...mea cupla :-). In that post I mentioned using delegates for multi-threading and said that you always had to use EndInvoke to avoid a memory leak...well not quite, what i actually do is use the excellent AsyncHelper class by Mike Woodring, this supports this syntax:

CalcAndDisplaySumDelegate d = new CalcAndDisplaySumDelegate(someCalc.Add);
AsyncHelper.FireAndForget(d, 2, 3);

No EndInvoke required as it takes care of this using a DynamicInvoke internally. Well, saves a few lines of code!

Delegate.BeginInvoke() and memory leaks...

Blog Author — Thu, 20 Nov 2003 02:54:00 GMT

I'm getting increasingly involved in the world of Asynchrony and have been doing a bit of digging about into some of the pitfalls of multi-threading.
I came across an interesting one today involving the use of BeginInvoke without a corresponding EndInvoke when using delegates...many books (including the one I'm using most right now, the excellent Programming .NET Components by Juval Lowy) mention that a good way to fire events asynchronously (i.e., not blocking) is to simple call BeginInvoke on the delegate - cool seems easy enough!
Well, reading around a bit, I discovered this by Mike Woodring which contains this statement: "Starting with the 1.1 release of the .NET Framework, the SDK docs now carry a caution that mandates calling EndInvoke on delegates you've called BeginInvoke on in order to avoid potential leaks. This means you cannot simply "fire-and-forget" a call to BeginInvoke without the risk of running the risk of causing problems. "
Now, I can't find the bit in the docs which says that, but another article at O'Reilly mentions the same thing... Well, based on this evidence, it would seem to be a good idea to avoid this potential leak, luckily Mike's stuff has a handily little helper class which supports the 'Fire and Forget idiom' - dead easy, now I can have lovely fire and forget calls without worrying about unintended leaks :-)

UPDATE: Found the doc reference, if you have MSDN it's here, if not the on-line version is here

Memory Leak - solved - or 'When STAThread Goes Bad'

Blog Author — Tue, 18 Nov 2003 07:40:00 GMT

I posted a little while ago about an odd 'Memory Leak' I was having...well, after several days of trawling (and annoying Data Wanta who's excellent email component I was mistakenly blaming - sorry!), I finally found the answer! Turns out that the little [STAThread] attribute above the main class of the Comand Line app isn't so inoccuous after all - Dave pointed out this thread - apparently there's some sort of bug in .NET which can lead to what is effectively a leak when using multi-threading in a STAThreaded application!

Multi-Threading

WPF for the Web guy…a story of pain, despair and learning

Threading update...what I actually do...

Delegate.BeginInvoke() and memory leaks...

Memory Leak - solved - or 'When STAThread Goes Bad'