Weird Stuff

Today I didn't go for a walk and now I feel terrible. I am wondering why this video is like it is. 

Subscribe to Computerphile.

This video has the missing bits and you can see a bit more clearly what's going on, but it's still way short of what I would call an explanation. See Burrows-Wheeler transform.

But it does sound like people are using it to measure relative entropy or something in genome sequencing, just as David Wheeler used it as a spam filter.See Thanks for BZIP2. 

Subscribe to Precision Health.

David Gifford - Library Complexity and Short Read Alignment (Mapping)

9:53 See Chris Mack on Career Planning and the Principles of Metrology. The relevance being that if you are trying to track the outbreak of a pandemic using a PCR test with a very high gain setting then you may be able to do a much better job if you take into account the assumptions you are making about the relative populations of molecules in the biological samples that are the subject of the PCR analysis. For example, if you are testing in a hospital where you know there are many patients who have the symptoms of the disease you should expect a different set of results that those you get from testing a population where nobody has the symptoms, and that gives you a way to spread your limited testing ability to maximise the amount you learn about the spread of the actual disease.

21:58 is where he begins to explain the transform. He starts by pointing out that the transform BWT(T) is the sorted list of all possible suffixes of the string T, ...

37:52 Maybe if he pointed out that sorting the last column returns the first column then it is more obvious how occ() and count() could be derived entirely from a traversal of BWT(T) [cf 42:45].

52:15 Mike Burrows surprised at how this thing has taken-off, and an observation that for searching the data you only need to sort on as many columns as the length of the strings you're searching for and a comment about FM-indexing.

1:04:30 Using backtracking to do nearest suffix matches.

I wonder how many of these very general-purpose algorithms are well-known outside of the community of people working on genome databases? Actually, the subsequent discussion of how much guesswork and finger-crossing is involved answers that for you. This is very uncommon sci-fi! See you next Tuesday!

Subscribe to MIT OpenCourseWare.

Toby has been digging around some British scientific history:

Here's one on The Great Exhibition, which I think Charles Lutwidge Dodgson attended as an undergraduate (he was 23 and in his first year at Oxford). See Life and Letters of Lewis Carroll, Chapter II:

I think the first impression produced on you when you get inside is one of bewilderment. It looks like a sort of fairyland. As far as you can look in any direction, you see nothing but pillars hung about with shawls, carpets, &c., with long avenues of statues, fountains, canopies, etc., etc., etc. The first thing to be seen on entering is the Crystal Fountain, a most elegant one about thirty feet high at a rough guess, composed entirely of glass and pouring down jets of water from basin to basin; this is in the middle of the centre nave, and from it you can look down to either end, and up both transepts. The centre of the nave mostly consists of a long line of colossal statues, some most magnificent. The one considered the finest, I believe, is the Amazon and Tiger. She is sitting on horseback, and a tiger has fastened on the neck of the horse in front. You have to go to one side to see her face, and the other to see the horse's. The horse's face is really wonderful, expressing terror and pain so exactly, that you almost expect to hear it scream.... There are some very ingenious pieces of mechanism. A tree (in the French Compartment) with birds chirping and hopping from branch to branch exactly like life. The bird jumps across, turns round on the other branch, so as to face back again, settles its head and neck, and then in a few moments jumps back again. A bird standing at the foot of the tree trying to eat a beetle is rather a failure; it never succeeds in getting its head more than a quarter of an inch down, and that in uncomfortable little jerks, as if it was choking. I have to go to the Royal Academy, so must stop: as the subject is quite inexhaustible, there is no hope of ever coming to a regular finish.

So that's what paid for the Natural History Museum! I don't share Toby's optimism about technology. I just see it as a terrible source of waste. Waste of a lot of extrememly talented people's time and affort to produce things that are used for a few months for some tawdry, mundane task and then discarded or just left to collect dust when the next flashy model turns up and works for another few months, ... It's still unhinged, and it's not just me that thinks this, I should probably read Charles Babbage's book, ...

Subscribe to Tibees2.

See Eric Weinstein on Great Science and Good Science.