A couple of weeks ago I researched all the different possible node sizes, from 32×32 all the way up to 256×256. There are three places where you store nodes: On disk, in memory and on the GPU. In memory you generally want small nodes so you don’t bloat the footprint (a pixel in memory take 4*sizeof( float ) = 16 bytes) and stay relatively cache-coherent. But small nodes generate a ton of disk IO which is costly. The optimal node size for the renderer is a non-issue since you can always bundle nodes together in to atlases and a pixel on the GPU takes only 4 bytes.
So the problem is disk vs. main-memory, which one should you optimize for? or can you find a workaround?
After spending a couple of days trying to come up with a good solution for minimizing disk IO, while maintaining small nodes (32×32 or 64×64), I finally gave up since all my solutions was immensely complex and the disk IO was still was pretty bad. Instead I decided to set the node size to whatever is optimal for disk and then try to work around the problems that would arise in memory. I knew the optimal node size for disk was either 128×128 or 256×256. What was surprising though was that 256×256 only generated about 60% of the IO requests compared to 128×128 where you naively would have expected ~25% and since the bandwidth and file-size goes up with 256×256 I decided against it and set the final node size to 128×128.
To solve the memory footprint problem I introduced nodes in different pixel precision. Doing it this way means that nodes that should only be displayed can stay in the same precision as on disk while nodes that should be edited can be converted to RGBA_FP (the main reason I use RGBA_FP is to spare myself all the fixed-point math headaches during development). If you start running low on memory you can start converting nodes back to file precision before you finally clean and unmap them.
A side effect of using large node sizes is that the “TOC” (Table Of Content) for each file gets a lot smaller which allows you to store hash values in them (somewhat similar to a .torrent file). This turns out the be extremely convenient if the user stores his data in the cloud since everything can then be fetched from a local cache file eliminating 99% of all downstream traffic, and since Amazon don’t charge you for upstream traffic you have effectively eliminated 99% of the bandwidth cost of using Amazon S3! (of course if the user change host, all data must be downloaded again but that should be quite rare).