|Fuck mixer taps
||[Nov. 11th, 2013|03:21 pm]
We've recently moved house, to a refitted Victorian tenement flat in Leith.
We're renting it from a lovely couple from Continental Europe, and this I
suspect is the reason for one of the few things that annoy me about the place:
that every sink in the flat is fitted with [mixer
Ordinarily this is merely a mild irritant, but occasionally (as happened this
morning), they drive me into a towering rage. Let me explain...
I'd taken out the contents of the food recycling bin, but a foul-smelling brown
gunge still coated the insides of the bin itself. I was therefore filling the
bin with a mix of bleach and hot water, the latter from the bathroom sink. The
sink was too small to fit the bin in, so I was filling a pint cup with hot
water from the sink and tipping it into the bin. Fortunately it's quite a small
bin. My attention lapsed for a moment, though, and the water overflowed, mildly
but painfully scalding my left hand. No problem: I could keep filling the hot
water with my right hand, while holding my left hand under the cold tap for as
long as it took to cool down. Except, oh, wait, mixer taps. Dammit. So I had to
turn off the hot tap, put down the cup, turn on the cold tap, and wait
uselessly for however long it took for my hand to stop hurting.
Except I had forgotten about the *other* problem with mixer taps: hysteresis.
When you turn off the hot water in a mixer tap system, you see, you don't reset
the tap to a safe state: a slug of hot water remains in the pipe, lying in wait
for the unwary. And so when I put my sore hand under the tap and turned on the
cold water, I was instead treated to a high-pressure dose of painfully hot water
onto the already painful area.
And then a few minutes later, while mentally composing this blog post and
muttering curses against the inventors of mixer taps and their descendents,
yea, unto the seventh generation, the **same thing happened to me again**.
In conclusion: fuck mixer taps. Fuck them right in their stupid single
non-parallelisable pain-causing water outlets.
This post is dedicated to elmyra, who labours
under the misapprehension that mixer taps are not only a superior
technology, but so obviously a superior technology that the only possible reason they
have not been universally adopted can be ignorance of their existence.
This entry was originally posted at http://pozorvlak.dreamwidth.org/177634.html. Please comment wherever and however is most convenient for you!
||[Oct. 24th, 2013|09:50 pm]
I hate making decisions. And I'm right to do so, as [the emerging body of evidence on decision fatigue](http://www.nytimes.com/2011/08/21/magazine/do-you-suffer-from-decision-fatigue.html?pagewanted=all&_r=0) makes clear. But sometimes you have to make a decision in situations where there's no obviously good choice: sometimes the differences between options are trivial, sometimes the differences are significant but the advantages and disadvantages are finely balanced, and sometimes you just don't have enough information to assess what those advantages and disadvantages are, but need to make a decision anyway so you can move on.
Some people advocate rolling a die or tossing a coin in this situation. These people are clearly less indecisive than me. Some people advocate tossing a coin, and if you catch yourself thinking "dammit, the other option would have let me do X" then you have discovered your hidden underlying preference and can go for that one. These people are also clearly less indecisive than me: I do that every time. Finely-balanced advantages and disadvantages; if there were no opportunity cost, there would be no decision to be made.
However, I have discovered a procedure that allows me to deal with many of these situations, and to do so *quickly* and with minimal stress. If you can't find a good argument for choosing one option over the others, look for a stupid reason instead. And do so in a consistent and general way, to minimise the mental effort required. The short version of my system is
1. Pick the red one.
2. If that doesn't work, pick the one with more cats.
3. If that doesn't work, pick the one with more dogs.
4. Give up.
The more detailed version is
1. Eliminate all choices that are less than maximally red. Red, of course, is the Best Colour.
2. If more than one choice remains, eliminate all choices which are less than maximally feline. More cats beat fewer cats, fluffy cats beat smooth cats, cute cats beat ugly cats, kittens beat adult cats.
3. If more than one choice remains, eliminate all choices that are less than maximally canine, subject to the rules above with the obvious formal modification applied.
4. If more than one choice remains, it is officially Too Hard and you are entitled to give up. Go to the pub and order the beer you haven't had before¹.
This may sound stupid, but inventing this procedure has had a noticeable positive effect on my life. It's quick to run through. It turns angsty and tiring vacillation into either a purely mechanical procedure, or a fun game of inventing reasons why abstract things are red, or catlike, or doglike. It works remarkably often: I can't remember when I last had to invoke Rule 4 (though it has had at least one notable success: see below). Hell, Rule 1 is enough most of the time. And, applied consistently over a long period, it causes my life to fill with things that are red, feline or canine, all of which make me happier.
*Q: Which of these two equally cute and fluffy kittens should you pick to take home with you?
Obviously these advantages are not tied to the specific steps listed above. Feel free to substitute your own favourite colour or animals, or to come up with different steps entirely. But I do recommend inventing a procedure like this one if you also struggle to make decisions.
¹ If there is more than one beer available that you haven't had before, drink them all in left-to-right order.
This entry was originally posted at http://pozorvlak.dreamwidth.org/177194.html. Please comment wherever and however is most convenient for you!
|Hymn to Ninkasi
||[Oct. 3rd, 2013|09:02 pm]
We've just moved house, which means that we're waiting for a new broadband
connection to be set up. While we're waiting for reliable Internet we've been
making our own entertainment, and by "entertainment" I of course mean "beer".
It's our first attempt at homebrewing, so we decided to walk before we tried
running and bought a [beginner's homebrew
Which made me think, as one does, of
[Ninkasi](http://en.wikipedia.org/wiki/Ninkasi), the Sumerian goddess of beer
and brewing, who every day used to brew beer for the rest of the Sumerian
pantheon. Beer was one of the crucial enabling technologies for the early
urban civilisations in Egypt and Mesopotamia: as well as being fun to consume,
it allows you to live in close proximity to other humans without dying of
cholera from contaminated drinking water. Beer brewing tech has [moved on a
since Ninkasi's day, but the essentials of the process remain the same:
germinate grains to turn the starches into sugars, soak in water to extract the
sugars, drain off the resultant liquid (the "wort"), mix with flavourings, ferment in a large vat, drink, enjoy. And since
today is [National Poetry
Day](http://www.poetrysociety.org.uk/content/info/npd/) (with a theme of
"water", no less), I thought it might be fun to update the Hymn to
Ninkasi. Dating from around 1800BC, it's one of the oldest
known recipes for beer. Here's an [academic translation](http://etcsl.orinst.ox.ac.uk/section4/tr4231.htm), and here's a [discussion, and a looser but more poetic translation](http://beeradvocate.com/articles/304).
> Given birth by the flowing water, tenderly cared for by Ninhursaja! Ninkasi, given birth by the flowing water, tenderly cared for by Ninhursaja!
> Having founded your town upon wax, she completed its great walls for you. Ninkasi, having founded your town upon wax, she completed its great walls for you.
> Your father is Enki, the lord Nudimmud, and your mother is Ninti, the queen of the abzu. Ninkasi, your father is Enki, the lord Nudimmud, and your mother is Ninti, the queen of the [abzu](http://en.wikipedia.org/wiki/Abzu).
> It is you who watches the included instructional DVD, and is soothed by the presenter's reassuring Australian accent and self-deprecating humour. Ninkasi, it is you who watches the included instructional DVD, and is soothed by the presenter's reassuring Australian accent and self-deprecating humour.
> It is you who puts a cupful of bleach in the plastic bucket, and fills it up with hot water. Ninkasi, it is you who puts a cupful of bleach in the plastic bucket, and fills it up with hot water.
> It is you who leaves the bucket to sterilise for half an hour, and then rinses it down carefully in the shower. Ninkasi, it is you who leaves the bucket to sterilise for half an hour, and then rinses it down carefully in the shower.
> It is you who carries the bucket through to the kitchen, and places it on the counter. Ninkasi, it is you who carries the bucket through to the kitchen, and places it on the counter.
> It is you who peels the backing from the thermometer strip, and sticks it to
> the outside of the bucket. Ninkasi, it is you who peels the backing from the
> thermometer strip, and sticks it to the outside of the bucket.
> It is you who upends the wort can into a saucepan of hot water so that it may flow more easily. Ninkasi, it is you who upends the wort can into a saucepan of hot water so that it may flow more easily.
> It is you who opens the can of hopped wort, and guards it even from the noble cats. Ninkasi, it is you who opens the can of hopped wort, and guards it even from the noble cats.
> It is you who dissolves the wort in two litres of boiling water, then adds the 1kg packet of fermentable and non-fermentable sugars and stirs vigorously. Ninkasi, it is you who dissolves the wort in two litres of boiling water, then adds the 1kg packet of fermentable and non-fermentable sugars and stirs vigorously.
> It is you who wonders what the hell the non-fermentable sugars are there for anyway, and tries to tell your girlfriend about [that Holsten Pils advert with Dennis Leary](http://www.youtube.com/watch?v=vmtxc-XGMSE), and remembers that she's too young to remember it. Ninkasi, it is you who wonders what the hell the non-fermentable sugars are there for anyway, and tries to tell your girlfriend about [that Holsten Pils advert with Dennis Leary](http://www.youtube.com/watch?v=vmtxc-XGMSE), and remembers that she's too young to remember it.
> It is you who tops up the bucket to twenty litres with cold water, stirs and checks the temperature. Ninkasi, it is you who tops up the bucket to twenty litres with cold water, stirs and checks the temperature.
> It is you who adds another couple of litres of cold water to ensure the temperature is within the range 21C-27C. Ninkasi, it is you who adds another couple of litres of cold water to ensure the temperature is within the range 21C-27C.
> It is you who takes the packet of yeast, tries to tear it open, realises there is no slit in the packet, and looks frantically for a pair of scissors. Ninkasi, it is you who takes the packet of yeast, tries to tear it open, realises there is no slit in the packet, and looks frantically for a pair of scissors.
> It is you who cuts open the packet of yeast with a Swiss Army knife and pours it into the bucket. Ninkasi, it is you who cuts open the packet of yeast with a Swiss Army knife and pours it into the bucket.
> It is you who curses when your girlfriend reminds you that you were meant to sprinkle the yeast evenly over the surface of the liquid. Ninkasi, it is you who curses when your girlfriend reminds you that you were meant to sprinkle the yeast evenly over the surface of the liquid.
> It is you who accepts all the blame if this whole thing goes wrong. Ninkasi, it is you who accepts all the blame if this whole thing goes wrong.
> It is you who slots the stupidly-named Krausen Kollar onto the bucket, and then fits the lid. Ninkasi, it is you who slots the stupidly-named Krausen Kollar onto the bucket, and then fits the lid.
> It is you who wonders why this system doesn't use an airlock, Googles to find out, and emerges no wiser. Ninkasi, it is you who wonders why this system doesn't use an airlock, Googles to find out, and emerges no wiser.
> It is you who hopes that you won't end up with partially-fermented beer all over your new kitchen floor. Ninkasi, it is you who hopes that you won't end up with partially-fermented beer all over your new kitchen floor.
> It is you who half-fills the hydrometer tube with the diluted wort and drops in the weighted bulb, to determine the beer's original gravity. Ninkasi, it is you who half-fills the hydrometer tube with the diluted wort and drops in the weighted bulb, to determine the beer's original gravity.
> It is you who attempts to read the specific gravity at the meniscus, but can't actually see the meniscus because beer is fizzy. Ninkasi, it is you who attempts to read the specific gravity at the meniscus, but can't actually see the meniscus because beer is fizzy.
> It is you who decides that accuracy to +/- 0.001 is probably good enough, and writes down your best guess in the Brewer's Log. Ninkasi, it is you who decides that accuracy to +/- 0.001 is probably good enough, and writes down your best guess in the Brewer's Log.
> [It is you who has only proceeded up to this point so far, and is copying the remaining steps out of the instruction booklet. Ninkasi, it is you who has only proceeded up to this point so far, and is copying the remaining steps out of the instruction booklet.]
> It is you who waits for four days, periodically checking that the temperature is within the range 21-27C, and then draws off another tubeful of liquid and measures the specific gravity with the hydrometer. Ninkasi, it is you who waits for four days, periodically checking that the temperature is within the range 21-27C, and then draws off another tubeful of liquid and measures the specific gravity with the hydrometer.
> It is you who checks the specific gravity daily until it has remained the same for 24 hours. Ninkasi, it is you who checks the specific gravity daily until it has remained the same for 24 hours.
> It is you who nervously taste-tests the beer, hoping that it has avoided infection or other problems. Ninkasi, it is you who nervously taste-tests the beer, hoping that it has avoided infection or other problems.
> It is you who sterilises the supplied heavy PET bottles, ready to receive the beer. Ninkasi, it is you who sterilises the supplied heavy PET bottles, ready to receive the beer.
> It is you who fits the bottling valve to the tube, and opens the tap. Ninkasi, it is you who fits the bottling valve to the tube, and opens the tap.
> It is you who fills the bottles with the beer. Ninkasi, it is you who fills the bottles with the beer.
> It is you who puts one sugar tablet into each 500mL bottle so that the beer may undergo a second fermentation, then caps them and inverts each one several times. Ninkasi, it is you who puts one sugar tablet into each 500mL bottle so that the beer may undergo a second fermentation, then caps them and inverts each one several times.
> It is you who wonders what was going on in that bit in *1984* where the old guy complained that half a litre of beer wasn't enough, seriously, you can barely tell the difference between half a litre and a pint. Ninkasi, it is you who wonders what was going on in that bit in *1984* where the old guy complained that half a litre of beer wasn't enough, seriously, you can barely tell the difference between half a litre and a pint.
> It is you who stores the bottles upright at a temperature above 18C (in Scotland, in October) for two weeks while the beer undergoes carbonation. Ninkasi, it is you who stores the bottles upright at a temperature above 18C (in Scotland, in October) for two weeks while the beer undergoes carbonation.
> It is you who invites your friends round to try the finished beer; it is like the onrush of the Tigris and the Euphrates. Ninkasi, it is you who invites your friends round to try the finished beer; it is like the onrush of the Tigris and the Euphrates.
This entry was originally posted at http://pozorvlak.dreamwidth.org/177124.html. Please comment wherever and however is most convenient for you!
|Lessons learned on my first Alpine trip
||[Sep. 23rd, 2013|12:17 am]
I got back from my first climbing trip to the Alps a bit over a month
ago. My friend Andy and I spent two-and-a-bit weeks in the Écrins
climbing routes graded Facile and Peu Difficile, and generally having a
blast. I'd wanted to go to the Alps for years, but I'd heard so much
about how hard and scary Alpine climbing was that I consistently failed
to get a trip together. This year I finally overcame my fears and
disorganisation and made it out there, and I found that low-grade Alpine
mountaineering (the stuff I was interested in, in other words) was much
easier, more fun and less scary than I'd been led to expect. I wish I'd
gone five years ago.
Some of the stuff I read beforehand was useful (in particular, I
recommend the BMC's DVD [Alpine
Essentials](https://www.thebmc.co.uk/alpine-essentials-dvd), which does
a wonderful job of demystifying Alpine mountaineering), but I was still
left with some misconceptions and gaps in my knowledge. Here's some of
what I wish I'd known back then; [the usual
### There is absolutely no need to go to the Dolomites first
People kept telling me that I should go to the Dolomites and do lots of
long rock routes before attempting the high Alps. I'm sure the Dolomites
are lovely, but this is nonsense. If, like me, you want to do low-grade
mountaineering on high mountains, go and do it. Experience of 10+-pitch
rock routes is not necessary; I doubt it's even very useful. Moving
together on scrambling terrain in big boots is a different skill.
### It's nothing like the books
Climbing memoirs and films concentrate disproportionately on the
super-hard routes and the times when Everything Went Wrong. It turns out
that there are plenty of easier routes too.
### Guidebook times are perfectly achievable
I received differing advice on this. All the books and DVDs stressed the
importance of completing routes within guidebook time, and only
increasing your grade/altitude/length once you were doing so. However,
most of the *people* I spoke to said that this was an unrealistic aim.
One guy even said that 1.5x guidebook time was a more reasonable target,
but that I (as a Slow Climber) should allow double. In the event, we did
almost all of our routes either within guidebook time or only a few
minutes over. On the one exception, the South Ridge of the Aiguille
Centrale de Soreiller, we took 4.5h versus 3h, but (a) the time quoted
was for the shorter variation of the route, and we did the longer
variation, (b) we deliberately decided to pitch the exposed summit ridge
rather than moving together, having assessed the glacier below and
deciding it would be safe to descend later in the day.
### It's not that scary
All the books said "you need a few days to get used to the sheer scale
of the Alps". I really didn't find this. Granted, the Écrins are not the
tallest part of the Alps, but the exposure levels were at most about
double what I'm used to from Scotland, and I felt pretty much at home.
In fact, I found fear-management much easier than on the average UK
roadside crag. If you can handle the Cuillin Ridge, you'll be fine.
That said, learning the [Litany Against
Fear](http://dune.wikia.com/wiki/Litany_Against_Fear) is not a bad idea.
It actually helps, and as an earworm it beats the hell out of *Brown
Girl in the Ring*.
### The days needn't be that long
Similarly with the oft-repeated advice that Alpine days can be really
really long. Our longest day was ten hours (although we stopped back at
the hut - it would have been more like 14 if we'd descended to the
valley the same day); I've done 13-hour winter days in Scotland, and
14-hour days in summer. Or 18-hour days if you count epics. There are
obviously plenty of very long routes in the Alps, but *you don't have to
do them*. Pick a shorter route and move fast.
I suspect that most advice to newbie Brit alpinists is aimed at
hot-headed wannabe Sheffield hardmen. The exposure's huge and the days
are long if you think that 10m of gritstone is a long route. Which
reminds me of the time I was climbing Curved Ridge in summer with three
friends, moving roped together, and two know-it-alls with Yorkshire
accents told us that they'd been climbing for thirty years and what we
were doing was "not a recognised rope technique". Hey, how about you (a)
fuck off back to Stanage, and (b) pick up a fucking book? I'm sure if
you ask nicely in the climbing shop they'll help you with the big words.
### You'll spend a lot of time downclimbing
The *voies normales* are, almost by definition, the easiest routes up
the mountains. Hence, if there were an easy route off the top, you'd
have climbed that instead. You may be able to abseil some sections, but
there's no guarantee of this.
### It's surprisingly warm up there
My previous experience of climbing in snow and ice had all been in
Scottish winter conditions, where your fingers are usually painfully
cold, touching the rock ungloved will chill you to the bone, hydration
tubes freeze solid, and if you stop for more than a minute you'll need
to layer up or dance about or both to keep warm. This was not the case
in the Écrins. I did most routes in just a base layer, with my thin
belay jacket coming out for summit stops or the occasional fixed belay
in the wind. My hardshell only got used during rainstorms in the valley,
and my outer gloves were entirely unused. Softshells were more useful:
in particular, my [Rab Sawtooth softshell
On the Barre des Écrins, a guide asked me "do you get many days like
this on Ben Nevis?"
"Oh yeah, we get some sunny days, even in winter."
"No, I meant with the wind!"
"Oh, right. In Scotland, we don't consider it windy if you can hold a
### Staying hydrated is hard
Non-freezing hydration tubes make it easier to take a drink without
stopping - and you will have very few stops if you're doing it right -
but we kept running out of water in the heat. The lack of stops also
means you can't do much to adjust your layering system if you get too
hot. On our first route - which took a mere 4.5 hours hut-to-hut - I
drank the whole of my 2L hydration bladder, then knocked back a 1.5L
bottle of water on my own back at the hut.
We also struggled to eat enough on the routes; we never properly hit the
wall, but we were definitely suffering from depleted blood-sugar on
several occasions. My normal strategy is to scoff chocolate biscuits and
sandwiches on belays, or eat while hiking, but this doesn't work when
you're moving together on class 3-4 terrain, need your hands to make
progress and can't spare the time to stop. I suggested filling our
drinking bladders with Gatorade or something similar to Andy, but
apparently when he tried that on a previous trip he lost a tooth.
### Lassitude is a real thing
I was astonished how little energy I had down in the valleys. The heat
sapped the power to do anything except lie about and drink tea.
### You'll do a lot of traversing
It turns out that
1. you have muscles in the side of your calves
2. they're used a lot when you traverse steep slopes
3. almost nothing else trains them.
### You'll need to switch very quickly between belayed climbing and moving together
File this one under "try not to stop for any reason" - you quite often
reach a spot where you can belay the leader over a tricky bit, but the
second wants to move off *immediately* once the rope comes tight. This
argues for the use of direct belays off spikes, a technique which
horrified me when I first saw it but to which I quickly became
### Fitness is useful, but you don't have to be an elite super-athlete
I had an ambitious training plan, involving half-marathons and marathons
and mountain marathons, but due to various injuries and illnesses and my
local gym closing down and other such excuses, I utterly failed to go
through with it. Consequently I headed out to the Alps well below my
usual level of fitness and carrying about 15kg of excess weight. About a
week before I went out, I ran 10km and got [delayed onset muscle
so long had it been since I'd done any running. And, you know what? I
was mostly fine. The walk-ins to the huts were hard, largely because we
were doing them in the heat of the afternoon (see above, "lassitude is a
real thing"), and I was pretty spaced out with tiredness on the descent
from the Barre des Écrins, but I managed. More fitness would definitely
have helped, sure, but lack of fitness wasn't (usually) the limiting
### Alpine star fields are *amazing*
Install a star-map app on your phone before you go. Trust me on this.
So what would be a really useful training plan for that sort of trip? I
suggest the following:
- Do as much hillwalking as you can. If it involves some scrambling,
all the better. Practice traversing steep slopes.
- Practice climbing easy routes in big boots.
- Practice *down*climbing easy routes in big boots.
- Practice climbing with a full bladder (once again, you don't want to
stop if you can avoid it).
- Do lots of long, grade I/II Scottish winter routes: the kind of thing
where you want to move together. This was the only part of this
training programme that I actually did, and I'm extremely grateful
- Practice your French (or the language of whatever country you're
visiting). High school was a long time ago for me, and it's
embarrassing asking "Parlez-vous Anglais?" all the time. Also, the
English-language guidebooks are selective and concentrate a lot on
the more aspirational routes; reading the local guidebooks will give
you more options.
tl;dr Alpine climbing is the Best Thing Ever. All the fun of Scottish
winter climbing without the hot aches.
![Me on the summit of La Grande Ruine](http://i.imgur.com/fEhXApRl.jpg)
*Me on my first Alpine summit, La Grande Ruine 3765m. [More photos here](https://www.dropbox.com/photos/c/oHOaHkMw1bxMaUM).*
This entry was originally posted at http://pozorvlak.dreamwidth.org/176715.html. Please comment wherever and however is most convenient for you!
|Jist tae let ye no
||[Sep. 7th, 2013|11:29 pm]
that wur in
ye wur prably
fer the party
They were great
This entry was originally posted at http://pozorvlak.dreamwidth.org/176385.html. Please comment wherever and however is most convenient for you!
|Srelipmoc ni esruoc tsrif a
||[Aug. 19th, 2013|09:10 pm]
If I ever design a first course in compilers, I'll do it backwards: we'll start with code generation, move on to static analysis, then parse a token stream and finally lex source text. As we go along, students will be required to implement a small compiler for a simple language, using the techniques they've just learned. The (excellent) Stanford/Coursera compilers course does something similar, but they proceed in the opposite direction, following the dataflow in the final compiler: first they cover lexing, then parsing, then syntax analysis, then codegen. The first Edinburgh compilers course follows roughly the same plan of lectures, and I expect many other universities' courses do too.
I think a backwards course would work better for two reasons:
What am I missing?
- Halfway through the Stanford course, you have a program that can convert source text into an intermediate representation with which you can't do very much. Halfway through the backwards course, you'd have a compiler for an unfriendly source language: you could write programs directly in whatever level of IR you'd got to (I'm assuming a sensible implementation language that doesn't make entering data literals too hard), and compile them using code you'd written to native code. I think that would be pretty motivating.
- When I did the Stanford course, all the really good learning experiences were in the back end. Writing a Yacc parser was a fiddly but largely straightforward experience; writing the code generator taught me loads about how your HLL code is actually executed and how runtime systems work. I also learned some less obvious things like the value of formal language specifications¹. Most CS students won't grow up to be compiler hackers, but they will find it invaluable to have a good idea of what production compilers do to their HLL code; it'll be much more useful than knowing about all the different types of parsing techniques, anyway². Students will drop out halfway through the course, and even those who make it all the way through will be tired and stressed by the end and will thus take in less than they did at the beginning: this argues for front-loading the important material.
¹ I learned this the hard way. I largely ignored the formal part of the spec when writing my code generator, relying instead on the informal description; then towards the end of the allocated period I took a closer look at it and realised that it provided simple answers to all the thorny issues I'd been struggling with.
² The answer to "how should I write this parser?" in an industrial context is usually either "with a parser generator" or "recursive descent". LALR parsers such as those produced by Yacc are a pain to debug if you don't understand the underlying theory, true, but that's IMHO an argument for using a parser generator based on some other algorithm, most of which are more forgiving.
This entry was originally posted at http://pozorvlak.dreamwidth.org/176246.html. Please comment wherever and however is most convenient for you!
|CouchDB - JSON and B-trees and REST, oh my!
||[Jun. 2nd, 2013|09:15 pm]
I've been learning about the NoSQL database CouchDB, mainly from the Definitive Guide, but also from the Coursera Introduction to Data Science course and through an informative chat with necaris, who has used it extensively at Esplorio. The current draft of the Definitive Guide is rather out-of-date and has several long-open pull requests on GitHub, which doesn't exactly inspire confidence, but CouchDB itself appears to be actively maintained. I have yet to use CouchDB in anger, but here's what I've learned so far:|
- The JSON is not by default required to conform to any particular schema, but you can add validation functions to be called every time data is added to the database. These will reject improperly-formed data.
- CouchDB is at pains to be RESTful, to emit proper cache-invalidation data, and so on, and this is key to scaling it out: put a contiguous subset of (a consistent hash of) the keyspace on each machine, and build a tree of reverse HTTP proxies (possibly caching ones) in front of your database cluster.
- CouchDB's killer feature is probably master-to-master replication: if you want to do DB operations on a machine that's sometimes disconnected from the rest of the cluster (a mobile device, say), then you can do so, and sync changes up and down when you reconnect. Conflicts are flagged but not resolved by default; you can resolve them manually or automatically by recording a new version of the conflicted object. Replication is also used for load-balancing, failover and scaling out: you can maintain one or more machines that constantly replicate the master server for a section of keyspace, and you can replicate only a subset of keyspace onto a new database when you need to expand.
- CouchDB doesn't guarantee to preserve all the history of an object, and in particular replications only seem to send the most recent version; I think this precludes Git-style three-way merge from the conflicting versions' most recent common ancestor (and forget about Darcs-style full-history merging!).
- The cluster-management story isn't as good as for some other systems, but there are a couple of PaaS offerings.
- Queries/views and non-primary indexes are both handled using map/reduce. If you want to index on something other than the primary key - posts by date, say - then you write a map query which emits (date, post) pairs. These are put into another B-tree, which is stored on disk; clever things are done to mark subtrees invalid as new data comes in, and changes to the query result or index are calculated lazily. Since indices are stored as B-trees, it's cheap to get all the objects within a given range of secondary keys: all posts in February, for instance.
- CouchDB's reduce functions are crippled: attempting to calculate anything that isn't a scalar or a fixed-size object is considered Bad Form, and may cause your machine(s) to thrash. AFAICT you can't reduce results from different machines by this mechanism: CouchDB Lounge requires you to write extra merge functions in Twisted Python.
- There's a very limited SQL view engine, but AFAICT nothing like Hive or Pig that can take a complex query and compile it down into a number of chained map/reduce jobs. The aforementioned restrictions on reduce functions mean that the strategy I've been taught for expressing joins as map/reduce jobs won't work; I don't know if this limitation is fundamental. But it's IME pretty rare to require general joins in applications: usually you want to do some filtering or summarisation on at least one side.
- CouchDB can't quite make up its mind whether it wants to be a database or a Web application framework. It comes by default with an administration web app called Futon; you can also use it to store and execute code for rendering objects as HTML, Atom, etc. Such code (along with views, validations etc) is stored in special JSON objects called "design documents": best practice is apparently to have one design document for each application that needs to access the underlying data. Since design documents are ordinary JSON objects, they are propagated between nodes by replications.
- However, various standard webapp-framework bits are missing, notably URL routing. But hey, you can always use mod_rewrite...
- There's a tool called Erica (and an older one called CouchApp) which allows you to sync design documents with more conventional source-code directories in your filesystem.
- CouchDB is written in Erlang, and the functional-programming influence shows up in other places: most types of user-defined function are required to be free of side-effects, for instance. Then there's the aforementioned uses of lazy evaluation and the append-only nature of the system as a whole. You can extend it with your own Erlang code or embed it into an Erlang application, bypassing the need for HTTP requests.
tl;dr if you've ever thought "data modelling and synchronisation are hard, let's just stick a load of JSON files in Git" (as I have, on several occasions), then CouchDB is probably a good fit to your needs. Especially if your analytics needs aren't too complicated.
This entry was originally posted at http://pozorvlak.dreamwidth.org/175882.html. Please comment wherever and however is most convenient for you!
||[Feb. 7th, 2013|12:41 pm]
Quaffing the last of my quickening cup,
I chuck fair Josie, my predatory protégée, behind her ear.
Into my knapsack I place fell Destruction,
my weapon in a thousand fights against the demon Logic
(not to mention his dread ally the Customer
who never knows exactly what she wants, but always wants it yesterday).
He sleeps lightly, but is ready
to leap into action, confounding the foe
with his strings of enchanted rubies and pearls.
To my thigh I strap Cecilweed, the aetherial horn
spun from rare African minerals in far Taiwan
and imbued with subtle magics by the wizards of Mountain View.
Shrugging on my Cuirass of Visibility,
I mount Wellington, my faithful iron steed
his spine wrought in the mighty forge of Diamondback
his innards cast by the cunning smiths of Shimano
and ride off, dodging monsters the height of a house
towards the place the ancients knew as Sràid na Banrighinn
The Street of the Queen.
Just wanna clarify that in lines 5 and 6 I'm not talking about the Growstuff customers, all of whom have been great.
|HiPEAC 2013 Berlin HLCGB
||[Jan. 24th, 2013|09:59 pm]
[Wherein we review an academic conference in the High/Low/Crush/Goal/Bane format used for reviewing juggling conventions on rec.juggling.]
High: My old Codeplay colleague Ally Donaldson's FAT-GPU workshop. He was talking about his GPUVerify system, which takes CUDA or OpenCL programs and either proves them free of data races and synchronisation-barrier conflicts, or finds a potential bug. It's based on an SMT solver; I think there's a lot of scope to apply constraint solvers to problems in compilation and embedded system design, and I'd like to learn more about them.
Also, getting to see the hotel's giant fishtank being cleaned, by scuba divers.
Low: My personal low point was telling a colleague about some of the problems my depression has been causing me, and having him laugh in my face - he'd been drinking, and thought I was exaggerating for comic effect. He immediately apologised when I told him that this wasn't the case, but still, not fun. The academic low point was the "current challenges in supercomputing" tutorial, which turned out to be a thinly-disguised sales pitch for the sponsor's FPGA cards. That tends not to happen at maths conferences...
Crush: am I allowed to have a crush on software? Because the benchmarking and visualisation infrastructure surrounding the Sniper x86 simulator looks so freaking cool. If I can throw away the mess of Makefiles, autoconf and R that serves the same role in our lab I will be very, very happy.
Goal: Go climbing on the Humboldthain Flakturm (fail - it turns out that Central Europe is quite cold in January, and nobody else fancied climbing on concrete at -7C). Get my various Coursera homeworks and bureaucratic form-filling done (fail - damn you, tasty German beer and hyperbolic discounting!). Meet up with maradydd, who was also in town (fail - comms and scheduling issues conspired against us. Next time, hopefully). See some interesting talks, and improve my general knowledge of the field (success!).
Bane: I was sharing a room with my Greek colleague Chris, who had a paper deadline on the Wednesday. This meant he was often up all night, and went to bed as I was getting up, so every trip into the room to get something was complicated by the presence of a sleeping person. He also kept turning the heating up until it was too hot for me to sleep. Dually, of course, he had to share his room with a crazy Brit who kept getting up as he was going to bed and opening the window to let freezing air in...
||most recent entries