Bon Statistiques: The Chernoff Bound

August 13th, 2009

The beauty of the Chernoff bound, like everything admirable, lies in the minimalism.

In the simplest form, it does this: Give me a random number between -1 and 1, with mean 0 and variance \sigma. The Chernoff bound can tell you themaximum possible probability of it being greater than a number \alpha, say p.

If the probability were any higher than p, the variance would need to be bumped up above the given \sigma. So this probability is intimately linked to \sigma.

Sketch of proof:
- Prove for single random variable
- Extend proof to two random variables.
- Invoke mathematical induction.

Ref: http://cseweb.ucsd.edu/~klevchen/techniques/chernoff.pdf

Granta Diary 2009

April 13th, 2009

Problem: The world has gone to the dogs and I can’t lay my hands on the Granta Diary for this year.
Briefly: Create my own, in LaTeX.
Solution: Life trained me to be a planning freak, and Granta gave me a diary that I could actually use. In this space I’ll put down the next installment of the glorious customized LaTeX journal that evolved from my calendars.

Culinary Carma

February 26th, 2009

Admire The Minimalist Mark Bittman, for the range and stripped bare minimalism of the recipes, for the super-comic videopods with their ever evolving titles sequence. I was to feature him in my links list and found — surprise surprise — he had stolen one of my recipes (where “stolen” means “o my god, he made a video of that, I do it all the time!”). Now I’m not saying it’s the wheatberries & scallions one (except that I go with oatmeal), because you’ll figure what I have for brisk breakfast on quick days when I’m all thumping energetic.

A part of his charm is the city kid in a kitchen simplicity of the recipes, and the gritty this is so good I can’t mess up even if I try attitude. Beyond that I have no idea why his podcasts interest me more than movie trailers. (Speaking of trailers, who’s this guy doing the gruffy voiceover in *all* of them? — That guy deserves the mallet — I’m so sick of him, good movies are good in spite of him, and movies that avoid him are good by default.)

Breakfast
* Oatmeal, onions, salt, water
* Bagel, creamcheese, bellpepper, mushrooms
* Oatmeal, frozen fruit, munkha & nuts, yogurt/milk

Recipes
* Zuke & Bellpepper
* Zuke & muchroom pasta
* Soy & wheat flour pudding/rice pudding

Metamorphic Copyrights

February 26th, 2009

A smart article deserves a convenient title: The Kindle Swindle?

Hypothetically, I get a copy of, say, “The Inconvenient Truth”, and pay for the book. So, in an Ideal World ™ that pays for reading the book once. Every time I read it, I should pay for it as for a fresh book, since it’s a fresh reading. If I read it aloud, it becomes an audio book, ka-ching for that. If I flip the pages quickly, it becomes moving words and pictures (does that version have pictures in it?) — hey, I’m watching a motion picture — ka-ching! Retroactively, I browsed it a bit at the bookstore, so it was an exhibited piece as if at a museum (arguable!) and it’s a work of art (hard to argue against!). Museum tix? Ka-ching! Going forward, it’s on my desk, and I work with loose sheets (ask Murphy, he things a writing board give Tina Fey glasses run for the make-me-look-cool money) — so yes, your honour, I used the tome as a paperweight. Ka-ching! That time when I played ping-pong because it seemed handy? Ka-ching. If I pack it with me on a flight? Umm, isolate the book and the planet from the distractions and you can clearly see: “Is it a bird? Is it a plane? No! it’s a flying book!” Mind it, we’re charging for it only as a toy plane! Ka-ching!

Really, the guy who wrote this article “The Kindle Swindle?” might charge you as if it were a REAL aircraft.

Some sense churns out from the article though: the insecurity of the creator in losing control of the creation. The insecurity of an author from yesterday facing the very different technical reality of today. Current patterns of media consumption demand an updated business model which ensures at least one thing: if you’re having more fun with your books now, the authors should be getting more money. That’s not sophisticated at all, but this is not a problem that I’m tackling tonight, so next thought:

Newspapers are dying because free online news are killing them. Free weather information on websites so ad defaced you suspect their owners sabotaged themselves. Noble news should be free in a democracy. Propagation of opinion should be free. Ugh. I don’t want to tackle this problem tonight either, for lack of time not of interest.

System Reset

January 19th, 2009

Problem When do I give up on a problem to seek an alternative solution, or a new problem? For example, I spent two days thinking through a math problem. A week or so for an information theory problem. A few weeks at a programming puzzle. A few years at my research topic. The sheer exultation at a solution makes it pointless to account for the time invested, but it needs to be subjected to kaizen. When should a light go off demanding a system reset?

Opinion A problem is solved. Persist at it. An error needs to be eliminated. Spotting the difference is challenging, and some questions that may help are:

  • Can I communicate (with whom?) my way through the problem? If yes, you’re staring at an error.
  • Have I decomposed my approach into smaller testable steps? If no, this is the first error. Break it down, and look for further issues.

Richard Bellman

October 19th, 2008

Stuart Dreyfus presents excerpts from Bellman, of dynamic programming fame, in RICHARD BELLMAN ON THE BIRTH OF DYNAMIC PROGRAMMING.

There’s a bit of research-related therapeutic talk here, and at least one provocative idea.

On logical scientific progress:

“Scientific developments can always be made logical and rational with sufficient hindsight. It is amazing, however, how clouded the crystal ball looks beforehand. We all wear such intellectual blinders and make such inexplicable blunders that it is amazing that any progress is made at all.

I strongly recommend the interesting study of these and related matters by Jacques Hadamard, the great French mathematician, in his book The Psychology of Invention in the Mathematical Field (Dover Publications, New York, 1945: paperback)”

On choosing your problem:

“Similarly, there are many questions that are difficult to answer, but hardly worth asking. The well-trained mathematician does not measure the value of a problem solely by its intractability. The challenge is there, but even very small boys do not accept all dares.”

“It is usually, if not always, impossible to predict where a theoretical investigation will end once started. But what one can be certain of is that the investigation of a meaningful scientific area will lead to meaningful mathematics. Inevitably, as soon as one pursues the basic theme of obtaining numerical answers to numerical questions, one will be led to all kinds of interesting and significant problems in pure mathematics”

Later Dogg

October 19th, 2008

That Voight-Kampff test of yours: Language

October 14th, 2008
  1. Tools
  2. Artificial Intelligence & Knowledge Representation
  3. Acoustics
  4. Computational Linguistics
  5. Experimental Linguistics & Psycholinguistics
  6. Philosophy
  7. Language-based Linguistics (as opposed to Concept-based)
  8. Compilers/PL

Fonts of Fantasy

October 12th, 2008

Problem: My thesis requires fonts from Star Trek and Bladerunner, but they’re TrueType and LaTeX does not play well with them.

Briefly: LaTeX is anal about the directory tree. So learn the right directory names and save yourself a bundle of trouble.

Solution: The LaTeX fundus package has Star Trek font; but turns out it doesn’t work with my Ubuntu. So I wrote a script to install a ttf in a local texmf tree. Note that the tex tree conventions
leave you have little flexibility in naming directories. So ${TEXMF}/fonts/tfm, ${TEXMF}/tex/latex/, ${TEXMF}/fonts/truetype, etc. need to be acceptable to tex.
Read the rest of this entry »

Speech Scholar

October 11th, 2008