RDBMS, to rewrite or not to rewrite… I got confused…

Mike Stonebreaker (Ingres/Postgres) seems to be confused as well…

First he said Google’s Map/Reduce was “Missing most of the features that are routinely included in current DBMS”, but earlier he said to ditch RDBMS anyway because “modern use of computers renders many features of mainstream DBMS obsolete”.

So, what’s the catch? Should we still use RDBMS or not? Or should we still develop technologies based on relational databases while Mike develops himself the technology of the new era? Maybe that was the message anyway…

My opinion:

MapReduce is not a step backwards, there are sometimes when indexing is actually slower than brute-force. And I’m not saying that on insert time the indexes have to be updated and so on, I’m saying in the actual search for information, if the index is too complex (or too big) it might take more time to search through the index, compute the location of the data (which might be anywhere in a range of thousands of machines), retrieve the data and later on, sort, or search on the remaining fields.

MapReduce can effectively do everything in one step, while still in the machine and return less values per search (as opposed to primary key searches first) and therefore less data will be sent over the network and less time will be taken.

Of course, MapReduce (as any other brute-force methods) is hungry for resources. You need a very large cluster to make it really effective (1800 machines is enough :)) but that’s a step forward something different from RDBMS. In the distributed world, RDBMS won’t work at all, something have to be done and Google just gave the first step.

Did we wait for warp-speed to land on the moon?! No, we got a flying foil crap and landed on it anyway.

Next steps? Many… we can continue with brute-force and do a MapReduce on the index and use the index to retrieve in an even larger cluster, or use automata to iteratively search and store smaller “views” somewhere else, or do statistical indexes (quantum indexes) and get the best result we can get instead of all results… The possibilities are endless…

Lets wait and see how it goes, but yelling DO IT than later DON’T is just useless…

UPDATE:

This is not a rant against Stonebreaker, I share his ideas about the relational model being far too outdated and the need for something new. What I don’t agree, though, is that MapReduce is a step backwards, maybe not even a step forward, probably sideways.

The whole point is that the relational model is the thesis and there are lots of antithesis, we just didn’t come up with the synthesis yet.

LSF, Make and NFS 2

Recently I’ve posted this entry about how NFS cache was playing tricks on me and how sleep 1 kinda solved the issue.

The problem got worse, of course. I’ve raised to 5 seconds and in some cases it was still not enough, than I’ve learnt from the admins that the NFS cache timeout was 60 seconds!! I couldn’t sleep 60 on all of them, so I had to come with a script:


timeout=60
while [ ! -s $file ] && (( $slept < $timeout )); do sleep 5; slept=$(($slept+5)); done

In a way it’s not ugly as it may seem… First, the alternative is to change the configuration (either disable cache or reduce timeout) in the whole filesystem and that would affect others. Second because now I just wait for the (almost) correct amount of time and only when I need (the first -s will get the file if there is no problem).

At least, sleep 60 on everything would be much worse! 😉

How close is nano-computing?

In September, Sunny Bains wrote Why Nano still macro? and since then I’m thinking about it once in a while.

Recently, a study in the University of California showed how to create a demodulator using nanotubes. So far there were advances in memory containers such as this and that and also batteries but all of them, as Sunny remembers, trying to build small structures following the design of big things.

Quantum computation nowadays have exactly the same problem, quantum effects in a classic assembly, big, clumsy and very expensive. If it was required a quantum effect (the transistor) to make classical computational cheap and available what will be required to make quantum computers cheap? A SuperString effect? Something messing around with the Calabi–Yau shape of the 6 additional dimensions?

Anyway, back to nanotech, building a nano-battery is cool but using ATPs as the primary source for energy would be much cooler! Using the available nano-gears and nanotubes to make a machine is also cool but creating a single 2,3 Turing machine (recently proven to be universal) would be way better!

Once you have the extremely simple processor like that, a nano-modem, some storage and ATP as food you can do whatever you want for how long you like inside any living being on Earth. Add a few gears to make a propeller and you’re mobile! 😉

Of course it’s not that simple, but most of the time to state that something is viable means exactly the same as to say that it’s classic as in boring and clumsy and expensive and brute force… well, you got the idea…

Yet another supercomputer

SciCortex is to launch their cluster-in-a-(lunch)-box with promo video and everything. Seems pretty nice but some things worries me a bit …

Of course a highly interconnected backpane and some smart shortest-path routing algorithms (probably not as good as Feynman’s) is much faster (and reliable?) than gigabit ethernet (myrinet also?). Of course, all-in-one chip technology is much faster and safer and more economic than any HP or IBM 1U node money can buy.

There are also some eye-candy like a pretty nice external case, dynamic resource partitioning (like VMS), native parallel filesystem, MPI optimized interconnection and so on… but do you remember Cray-1? It had wonderful vector machines but in the end it was so complex and monolithic that everyone got stuck with it and never used it anymore.

Assembling a 1024-node Linux cluster with PC nodes, Gigabit, PVFS, MPI etc is hard? Of course it is, but the day Intel stops selling PCs you can use AMD (and vice-versa) and you won’t have to stop using the old machines until you have a whole bunch of new ones up and running transparently integrated with your old cluster. If you do it right you can have a single cluster beowulf cluster running alphas, Intel, AMD, Suns etc, just bother with the paths and the rest is done.

I’m not saying it’s easier, nor cheaper (costs with air conditioning, cabling and power can be huge) but being locked to a vendor is not my favourite state of mind… Maybe if they had smaller machines (say 128 nodes) that could be assembled in a cluster and still allow external hardware to be connected having intelligent algorithms to understand the cost of migrating process to external nodes (based on network bandwidth and latency) would be better. Maybe it could even make their entry easier to existent clusters…

Geeks United! It’s time to recycle!

It’s time to recycle using your hand craft abilities!

Computer Chip Trivet

Don’t you know what to do with those old computer chips laying around? What do you think about a stylish trivet? Instructions are simple to follow: all you need are some computer chips , grout, adhesive, and a tile square.

Once you’re all finished, you’ll have a nicely geekified trivet for all your hot stuff.

You’ll really impress your geeky friends with this genuinely useful kitchen tool that you can make: a trivet built out of old computer chips.

Follow this link for full instructions.

Hard Drive Wind Chimes

The drive platters themselves are also quite remarkable: precisely made aluminium patters with a surface not unlike recording tape. The disks make a lovely clear note if you strike them, so it was only natural to make them into a set of wind chimes.

An interesting side effect is that the shiny shiny platters reflected little spots of light into the house. Naturally, if you have cats, they’ll love it too.


Follow this link for full instructions.


Hard Drive Picture Frames

So, you’ve disassembled hard drives, taken the magnets out, made wind chimes out of the platters, and so on. One thing that you might have left over is a set of printed circuit boards. Funny shaped printed circuit boards, with holes in them.

Here’s how to turn those leftover PCBs into fabulous geek-chic picture frames.

It’s done! Hang it on something ferromagnetic!

Here’s a completed picture frame, hanging on a wire bookshelf.

Follow this link for full instructions.

Keyboards for everybody

Are you planning change your keyboard and you don know which one choose? Maybe it could help you.

Are you old-fashioned? Full instructions for how to convert a 1980s era “clicky” IBM keyboard into an input device that would look at home in a movie adaptation of a Jules Verne novel are available on The Steampunk Workshop

 

For those who likes more futuristic stuff, the Bluetooth Laser Virtual Keyboard could be a great idea.

If you want to impress your colleagues and friends, you could use the Das Keyboard – 100% blank.

Windows user? Try the Microsoft Wireless Entertainment Desktop Keyboard 8000, with a magnetic dock for you wireless mouse and a bright Windows key.

Are you travelling all the time or maybe you just want to use your blackberry with a keyboard? So, the Elkensen Fabric Keyboard is perfect for you.

They have both wireless and wired version.

So, which one?