Selection Biases in Real World Data

So a web-page depicting the average SAT score by state has been making the rounds a little bit in Iowa. Apparently Iowans are so smart! But something seemed fishy about the data...oh, right the participation rate is 3% in Iowa, and in general states with a lower participation rate have better scores.



Now, if you take the ACT data for 2010, Iowa is around 15th, but with a much higher participation rate.



Look at Maine! Maine is ranked dead last, 51st, on the SAT list, but 5th on the ACT list, up there with a lot of East Coast states whose students didn't do so well on the SAT list. New York is 46th on one list and 4th on the other.

Why the difference in Iowa's rank? And why the huge difference in Maine's rank?

Biases in the test-takers is likely to be at work. ACT has its mothership in Iowa City, meaning that Iowa is likely to be pushing the ACT for political-economic ties. In my experience, only seasoned test-takers in Iowa take the SAT. The ACT is "good enough" for most people, but because of the additional practice and the inherent variance of the tests, you can get a higher personal best simply by more tests (personal note from an MIT grad: I took each test 6 times, once per year from 7th grade to 12th grade). Thus it is likely that Iowans hoping to get into a selective school are more likely to take the SAT. A similar effect is probably at work with the ACT in states like Massachusetts --- lower participations rates in these states are being caused by forces which select for the best test-takers and smartest students.

Making the SAT difficult to take in Iowa might help the state look the best in the SAT rankings, even though the ACT is HQ'ed in Iowa City. But most of all, reading too much into improperly gathered rankings is dangerous.

The Design of Everyday Diabetic Things

I figured I'd talk a little bit of the design of things used to treat insulin-dependent diabetes mellitus (IDDM). The equipment features interesting trade-offs between ease-of-use, safety, cost, effectiveness and comfort. There are two main sets of tools: one for checking glucose and one for injecting insulin.

The Glucose Meter

There are many different glucose meters, but I use a Bayer Contour, because that's what the doctors suggested. The meter requires a disposable strip every time I want to check my blood sugar. I insert the strip into the meter, and then hold a drop of blood to the test strip intake.


The gadget is the meter, the plastic container (above the gadget) holds the test strips, the lancer is used to prick fingers (I guess they didn't want to call it a finger pricker for whatever reaon), the mesh compartment holds microlets (finger prickers), and I have some alcohol swabs in the little compartment.


The meter with a test strip inserted. In order to get a reading I have to hold a drop of blood to the little chamber at the bottom of the strip.

What the meter looks like after a reading. I'm using mmol/L, not mg/dl, which is why the reading is so low. A person with blood glucose levels of 5 mg/dl would be in a coma.


The lancing device (finger pricker) with the cap off, along with an uncapped microlet and a capped microlet. You have to twist the cap off the microlet in order to use it; if a child swallows a capped microlet, the plastic is strong enough that the cap shouldn't come off. I think. I'm not really sure, and I'm not planning on running tests, but if you know a child who would be interested in swallowing one, please feel free to contact me.

The amount the microlet penetrates into your skin is adjustable via screwing the cap on the lancing device. Diabetics who prick their fingers a lot develop callouses, so depending on the thickness of the skin, the needle may need to penetrate farther.


I had initially thought the meter was really nice. They're really inexpensive, have some fancy features like alerts, and it is very easy to use. However, much like printers being cheap and ink being expensive, the meter is cheap but the microlets and test strips are very expensive. The microlets and strips cost about a dollar a piece, and IDDM people are supposed to check their blood sugar at least four times a day. You're also supposed to change the microlet (the needle used to prick your finger), as the microlet gets worn after use. In practice most type 1s I know only change the microlet once per day. Anyways, the $20 meter consumes over $1,000 of test strips per year (my insurance covers the use of about 1,600 test strips per year). Right now I'm checking my blood glucose levels about 7 or 8 times a day, since I'm still getting used to everything, but eventually it should go down to four or five times a day. If my insurance didn't cover the strips, I'd almost certainly use a meter with a strip disk, since they'd be cheaper long term.

Overall I'm pretty happy with the meter. It is a little bulky, so I'll probably pick up a smaller one that I could carry with me, but the fact that it was free and the strips are mostly covered by my insurance makes me happy.

Insulin pens

Once per morning and before every meal I have to give myself insulin. There are two types of insulin I use. Lantus, which acts very slowly over the course of about 20 to 24 hours, is called a basal insulin. I use Lantus once per day in the morning, though eventually I'll probably have to split it into a day and night dose. Apidra, which acts very quickly, is called a bolus insulin. I take Apidra before meals; the exact dose depends on the carbs in my meal, my current blood sugar, and the amount of sugar I expect to burn. The location of injection has an important role in how fast my body absorbs the insulin. Injecting in the thighs makes absorption quite slow, while the belly area absorbs quickly. I normally use my thighs for the slow-acting, and my belly for the fast acting. I have to rotate the exact site location, or else scar tissue builds up which can make absorption difficult.

The biggest danger with the pens is that I accidentally confuse the two types of insulin. While the pens are visually distinct, with different colors, I actually wish they were physically distinct --- in other words, the Lantus pen was larger than the Apidra pen. I'm not too worried about confusing the two, since I keep them in different spots, but it would be nice if they were more difficult to confuse.


I have my two pens with two sealed needles on the kitchen table. The pens are disposable; after the pen runs out of insulin, I throw them away. Before use, the pens have to be stored in a fridge. After use, the pens are good for 28 days. I write the date the pens expire after the first use, so I can easily track when I'm supposed to throw them away. The pens are small enough to fit in my pocket.


This is the dial on one of my pens. I choose the number of units to inject by rotating the dial.

This is where the needle screws onto the pen top, with the number of units of insulin units left in the pen displayed.

This is the needle after the seal is taken off. There are screw threads on the inside that screw onto the top of the pen. The needle is inside the plastic container, so it is quite difficult to prick your finger with this. If the needle falls, it won't jab anything.


The needle has two caps, an outer and an inner. The inner cap is color-coded to the size of the needle. Different injection sites require different needle sizes.

A 8mm and a 4mm needle attached to the pens. Before injection, two units are supposed to be "burned," or injected into garbage. Burning the units insures that no air is in the needle. My Apidra pen always has a small amount of pressure, which is why there is a drop of insulin on the end.

I think the pens have a phenomenal design. First of all, the screw cap allows the use of different needles. This is important because, as I previously mentioned, different sites require different needle sizes, and because the needle wears after use, a new needle is required every time. The screw cap doubles as a safety measure; you have to push and screw the needle at the same time to get the cap on, which kid-proofs the pen to a large extent. With a child, the real danger isn't that a kid plays with the needle; the real danger would be a kid injecting themselves with insulin. The screw dial also kid-proofs the pen to a certain extent. An additional feature is that as you inject, the screw dial resets to zero. This means that the default state of the pen is going to be to inject no insulin.

The injections are almost pain free. Insulin injections are subcutaneous, compared to vaccinations which can require injecting into a muscle. I barely notice them afterwards, which has contributed to how quickly I've gotten used to them. The needles and pens, while expensive, are also (almost completely) covered by my insurance.

As I stated earlier, the devices feature some interesting trade-offs. Overall, I've been very impressed by how little pain I've experienced. While I still wouldn't wish IDDM on anyone, it isn't as difficult to deal with as I had initially envisioned. However, my pancreas is still working at some fraction of capacity, so I'll have to wait and see if I think the devices are effective enough when my pancreas produces no insulin.

Diabetes FAQ

I'm getting a lot of questions about type 1 diabetes; I figured a blog post would be in order.

What is diabetes? What is the difference between the types?
The two types of diabetes mellitus are actually quite different, though both diseases relate to sugar and sugar metabolism. Type 1 diabetes mellitus, which I have, occurs when the immune system attacks the portion of the pancreas that produces insulin. Insulin is necessary for the body to metabolize sugar correctly. Type 2 diabetes mellitus occurs when the body doesn't respond correctly to insulin. Type 2 usually occurs in older, overweight individuals that have eaten a lot of sugar.


What's up with the mellitus at the end of diabetes mellitus?
There is another form of diabetes called diabetes insipidus, or DI. Mellitus means sweet, whereas insipidus means tasteless. The terms relate to the fact that urine from a person with diabetes mellitus, or DM, is sweet (due to the sugar flushed out of the body), whereas urine from a person with diabetes insipidus is tasteless, or tastes like whatever urine tastes like. My high school biology teacher once told me that to diagnosis whether problems were DM or DI, people drank the urine. Is it true? I have no idea.

The symptoms of DM and DI are similar, but the causes and treatments are very different. Diabetes, in general, usually refers to diabetes mellitus.

What does having type 1 diabetes entail?
Essentially I have to give myself insulin instead of relying on my pancreas to produce it for me. This means that before eating anything with carbohydrates I have to give myself insulin. Anything with sugar or starch is high in carbohydrates, so foods like potatoes, candy, and fruity drinks require giving myself more insulin than meat or low-sugar foods.

Do type 1s ever develop type 2 symptoms?
"Very rarely." is the response I've received from doctors.

Isn't type 1 called juvenile-onset diabetes?
Onset of type 1 can occur later than age 40, and thus the terms juvenile or juvenile-onset are no longer preferred. Type 1 is often called insulin-dependent diabetes mellitus or IDDM. I was 26 at onset, and I know another type 1 who was diagnosed when he was 29. Type 1 is actually "better" when it occurs later in life; when kids are growing it is difficult to find the correct amount of insulin to use.

You have to give yourself a shot before you eat almost anything? Does it hurt? How much insulin do you use?
I do have to give myself a shot before every meal. I also give myself a shot in the morning of long-acting insulin that stays in my system for about 24 hours. The shots hardly hurt at all, and I have very quickly gotten used to them. I have also gotten used to pricking my finger to check my blood sugar. Pricking my fingers was worse at first, but is no longer a problem.

Right now my pancreas is still producing some fraction of the insulin that would be expected. However, my body will continue to attack portions of my pancreas, and eventually it will produce only trace insulin. I'll have to inject more insulin at this point.

The amount of insulin I use depends on the amount of sugar in the food I eat. I have to match my insulin dose to the amount of sugar in the food I consume using a carbohydrate-to-insulin ratio. The ratio will change as my pancreas shuts down, but it should stay the same afterwards.

What are the symptoms of being type 1 diabetic?
The insulin shots are to keep my blood sugar at roughly the level of a normal person's blood sugar. There are no symptoms if my blood sugar stays in the normal range. Problems arise if my blood sugar is too high or too low.

If my blood sugar goes too low, I notice it immediately, and I have something that has a little bit of sugar in it. I'm trying to find something with sugar that I don't like --- giving myself a reward when I don't properly manage my blood sugar seems like a bad idea. When my blood sugar is too low, my head feels a bit weird and my muscles jerk instead of operating smoothly. If my blood sugar goes way too low, very bad things can happen, such as fainting. Having low blood sugar is dangerous short term.

If my blood sugar goes too high, then I usually have trouble focusing. If it stays too high, then I usually itch. The itching is a personal symptom, which many people do not have. If it stays too high for too long, I can develop more symptoms, from the mundane like increased acne and increased thirst to the severe symptoms of weight loss, muscle aches and very blurry vision. My body has acclimated to high blood sugar, so it is difficult for me right now to notice when my blood sugar is high.

If my blood sugar is too high long-term, there are many complications for my eyes, kidneys, heart and lungs.

How did you find out?
Over the past three months at work, I've had tremendous difficulty focusing. I've also noticed that I've had increased amounts of acne.

During winter vacation, I spent a week in Boston. My hands started to itch, and while I didn't think too much of it at the time I did see a doctor when I was home in Iowa a week later. I starting having the more severe symptoms, including increased thirst and blurry vision. I thought my vision was blurrier than normal, but I've also needed glasses for a while, so I didn't really notice it. Then my muscles starting being sore when I walked up the stairs. I searched for the symptoms using my search engine of choice and there it was: diabetes. I saw the doctor the next day. My blood pressure was normal, my heart rate was normal, and my blood sugar was 4x the value it was supposed to be. Based on my age, weight and blood sugar, it was an automatic type 1 diabetes diagnosis. I had also dropped 2.5 kilos (about 5 pounds) in the week between the two doctor appointments. The doctors said that the decreased exercise (I ride my bike / walk a lot in NL) combined with my root beer, birthday cake and chocolate diet caused a huge spike in my blood sugar.

How does exercise work?
Blood glucose levels drop automatically during exercise, so I am supposed to only give myself a half dose of insulin at meals before I work out. After I get my levels stabilized I think I am going to join a gym.

Can you still drink alcohol?
Yes, but alcohol is tricky. Most alcoholic drinks are just fermented sugar, and consuming small amounts of sugar over a long period of time does not match giving a single injection of insulin, which works well for a meal. Additionally, being intoxicated is much more dangerous now than before. I didn't consume alcohol in large quantities before, just a beer every now and then, but I think I am going to consume even less now.

This all kinda sucks.
That is not a question, but yes, it is unfortunate. However, it turns out that people can adapt very quickly to adverse circumstances. I can tell that I am going to be able to live a normal life sans some special things required for eating food and exercise.

How common is type 1 diabetes?
In northern European populations, the rate is almost 1%, or 1 in 100 people (see the study in the next question). In America the rate is probably closer to 1 in 200 or 250 people (see the Wikipedia article). I haven't found great information on the rest of the world.

Is the disease genetic?
There are some very interesting studies on this question, particularly a study on Finnish twins. When a twin of an identical pair had type 1 diabetes, there was roughly a 50% chance that the other twin would also have the disease. There are multiple genes involved, which make occurrence complicated to predict. The 23andMe page on type 1 diabetes is actually a really good resource on the genes and appropriate studies.

Type 2 is also somewhat genetic, but is much more influenced by the environment, especially diet and exercise.

If your pancreas isn't working does that mean you don't need it? Can I stab it?
Only a portion of my pancreas isn't working; I still need some of the other portions. No, you may not stab my pancreas.

Do you expect that anyone is going to develop a cure in your lifetime?
No. It wouldn't surprise me if someone came up with a vaccine, but I doubt there will be a cure. Drug companies aren't very interesting in curing things, they care more about treating symptoms, which is a better income stream.

What was treatment like before insulin?
Controlled starvation was the "treatment" before insulin. I use quotes because nothing was actually treated, symptoms existed, but people would live longer. The life expectancy after diagnosis without starvation was about six months; it could increase to a few years with controlled starvation.

After insulin was successfully extracted in 1921 (the extraction technique's inventors won the Nobel prize in 1923), treatment was an insulin injection before any meal. I would recommend reading the book Breakthrough, though make sure you read the book's afterword to understand that some of the non-scientific portions of the book are fictionalized.

What can I do?
The most important thing you can do is take care of yourself so that you are less likely to develop type 2 diabetes. Eating foods with less added sugar, eating lots of fruits/veggies, having a balanced diet, and getting enough exercise are very important.

If you like you can donate money to a foundation like the JDRF or ADA, but I think being supportive of scientific research and politicians who support research is more important.

Finally, just treat anyone who is diabetic just like a normal person with specific dietary restrictions. Try not to confuse type 1 with type 2; while both types share the label of diabetes mellitus, the two types can be very different. If a diabetic suggests eating at a different time or a different place please accept.

Oh, and be very careful about brunch. We the type 1 diabetics hate brunch. Well, mostly hate brunch. It is too much food of the wrong type at the wrong time.

This is all really interesting, where can I find more information?
I would suggest browsing the JDRF website first, then maybe the ADA website. There is always the Wikipedia.

Worst Birthday Present Ever

I found out I have Type 1 Diabetes Mellitus just after my 26th birthday. I am fine, please don't worry, it is just going to be a huge lifestyle adjustment in the coming months.

researchresearchresearch

In case you're wondering what is happening in my life, please see the title. No new events, no crazy nights, no fun vacations. Until December. When hopefully things will pick up.

I should get a hobby. That involves people. Hmmmmm.

LISP at work!?!?!

I get to use LISP at work! A standard CAD program, Cadence's icfb, uses a SKILL-based API. SKILL has a LISP-like syntax. I need to automate creation of a large number of cells, and the SKILL API is the easiest way to do that. This is going to be awesome!

Kwality

Stay classy, Adobe.

Science disappoints

Nothing makes me distrust a journal as much as ignorance of the state-of-art.

From a 2008 article in Nature Photonics!:

To date it has been thought that APDs are unable to measure the number of photons in a pulse of radiation or a short time interval.

And yet, from an article published four years earlier about APDs, we see:

A technique has been developed and tested that permits estimation of the photon number involved in a detection process.

Wow. Just wow. I would have thought the reviewing process for Nature Photonics would be rigorous. I guess you can't believe everything you read.

Open source!

I'm going to be creating an open source library with all of my models - woohoo! Now if only I could decide on which license to use - current contenders are BSD, MIT, GPL and apache. Hmmmm.

Makefile for \LaTeX

Due to some recent discussion on my previous LaTeX post, I thought I'd share the biggest advantage I think LaTeX has over WYSIWYG editors like MSWord: interaction with text tools.

First, let me paint you a few scenarios. You have images that you want to use in multiple places, like presentations, papers, and maybe even that PhD thesis you are planning on writing in a few years. However, the image format is going to vary from document to document; some documents, like presentations, call for much larger fonts, whereas your PhD thesis is going to have a clipped image size compared to any papers you'll publish. You want to be able to quickly reconfigure a set of images to have the same look and feel. You also want to view, easily, what processing you used on the image several years from now after you've forgotten everything you did.

Another scenario: a person is writing a book chapter and comes to you about using a variation of one of your images that she has seen in a paper of yours. She asks you to produce a variation of it.

Third scenario: one of your colleagues discovers a bug in an analysis script. You need to quickly track down all data that uses this script, and see if it affects any of your published data.

Fourth scenario: you're trying to remember exactly how you analyzed a set of data, but you performed the experiment several months back and can't remember the exact method you used to produce a figure in one of your presentations.

Fifth scenario: you're modifying a script that creates a figure, and you're concerned about comparing the different possibilities and still getting the final version correct. How do you track all the changes?

These scenarios illustrate three important ideas when publishing data: reproducibility, traceability, and configurability. I am going to share my method of (mostly) solving these problems using LaTeX and the make utility.

Make is a text-based utility that is normally used for dependency tracking in large programs. Back in the day, when compiling took a lot more time than it does now, it was important that code was only compiled when it was necessary. When a program called a library, it wasn't necessary to re-compile the library every time the program was changed. The make program is used to track these dependencies, but it is actually general enough to track a lot of dependencies, such as images in a LaTeX file.

Let me give you an example. Say you have a LaTeX file that includes two images: A.jpeg and B.jpeg. Both are created by eponymous scripts A.exe and B.exe. A.exe, though, relies on a lot of really complicated processing performing by complexProcess.exe. You also want to have the same look and feel for both images.

This is quite simple to accomplish if you are familiar with the make utility. You can create a Makefile that looks like the following, which tracks the dependencies of your data and processing:

A.jpeg: A.exe tempFileFromProcessing
    ./A.exe

tempFileFromProcessing: complexProcess.exe experimentalData
    ./complexProcess.exe

B.jpeg: B.exe
    ./B.exe

viewPdf: pdf
    acrobat paper.pdf

pdf:  paper.tex paper.bib
    latex paper
    bibtex paper.bib
    latex paper
    latex paper
    dvipdf paper.dvi

This looks complicated, but it is quite simple. The Makefile has the structure

makeItem: dependencyOne dependencyTwo
    commandOne
    commandTwo

To create makeItem, make checks that the dependencies are met. If makeItem and the dependencies are files, then make checks if the dependency files were last modified after the makeItem file, and will only run the commands if this is the case. In the Makefile example, A.jpeg depends on A.exe and tempFileFromProcessing. If either A.exe or tempFileFromProcessing has been modified after A.jpeg, then make will run ./A.exe (which should create A.jpeg). Similarly, tempFileFromProcessing depends on the processing script, complexProcess.exe, and experimentalData. If either complexProcess.exe or experimentalData has changed after the temp file, the processing will be re-run. This allows you to cache or save complicated processing, but still gives traceability into what processing occurs.

Making the file B.jpeg is simpler to understand, as it just depends on B.exe. If B.exe has changed after B.jpeg, make will run ./B.exe, which should create B.jpeg.

The same thing can be done with tables or experimentally derived values by having LaTeX include another .tex file that is generated by a script.

Getting LaTeX and BibTeX to generate a file can be a bit complicated. Using make erases this problem, as it does the same thing every time. To make the pdf, you'd type "make pdf" which would cause make to run latex, then bibtex, then latex, then latex again. You could type "make viewPdf" which would also tell adobe acrobat to show you the PDF file.

If you need to see which images depend on a specific script, you can have the script print out caller information (or use a logging utility) to check what data is passing through any buggy scripts. Just change the script and re-make the pdf, and you'll have all the contaminated data calls.

Okay, so this fixes two problems, namely reproducibility and traceability. It also makes it easy to change the file and see the results. You can change which experimental run you use for experimentalData, for example, to see how your A.jpeg varies based on which run you're presenting. But what about configurability? How do you make the plots look the same for each document, but use the same plotting scripts across documents?

You can accomplish that by using a convention for look-and-feel in the figure generation files. For example, you can pass a file to all of your scripts that contains information like the font, font size, and figure size to create. This allows easy configurability.

So there you have it - the biggest strengths of LaTeX compared to programs like MSWord. You can create documents that use the same plotting scripts, but with different look-and-feels, for publications, presentations, posters and theses. It is easy to trace the source of plots when you need to come back to that publication in four years but you forgot the exact processing. You get caching of complex processing. Finally, it is easy to track down what published data has images run through contaminated scripts. The fact that LaTeX is compiled gives you superior traceability, configurability, and reproducibility of your data compared to WYSIWYG editors like MSWord.

Compacting Conference Papers in LaTeX

I have a conference submission due next week with one page of text and two pages of figures. I had to compact this submission like crazy, so I thought I'd post my notes in case anyone else finds them helpful. "Anyone else" includes a future me.

Margins

I used the geometry package to get my margins correct.

\usepackage[left=0.5in,right=0.5in,vmargin=0.8in,nohead,nofoot]{geometry}

Title

Unfortunately LaTeX doesn't seem to handle compacting titles, authors and affiliations very well. My start looks like this:

\centerline{\Large \bf Title Goes Here}
\medskip
\centerline{Author~Name~One, Author~Two$^{\rm \dagger}$, and
  Author~Three$^{\rm \dagger*}$}
\centerline{Primary Author Institution, Address, Country}
\centerline{ phone: +1-123-456-7890, fax: +1-234-567-8901, email:
  user.name@institution.edu }
\centerline{$\dagger$Affiliation $*$Affiliation}
\bigskip

%% Text starts here

Figures

I used the vspace command quite liberally to shrink the space between the pictures and the captions. An example follows. The "[h!]" command after the "\begin{figure}" tells LaTeX to try really hard to keep the same ordering as in the .tex file.

\begin{figure}[h!]
\vspace{-25pt} %% moves figure closer to previous caption
\centering
\includegraphics[width=3.5in]{figure_file.pdf}
\vspace{-5pt}  %% moves the caption closer to the figure
\caption{\textbf{Figure title} - a detailed description of the figure.}
\label{fig:block}
\end{figure}

Equations

I embedded my equations in a figure to save space. I also used the vspace command to help with space between equations.

\begin{figure}[h!]
\centering
\vspace{-20pt}
\begin{equation}
\label{eq:one}
\frac{\partial c}{\partial t} = D\frac{\partial^2 c}{\partial x^2}
\end{equation}
\begin{equation}
\label{eq:two}
{\rm SNR} = 20\log_{10}\left(\mu\right) 
\end{equation}
\begin{equation}
\label{eq:three}
\Delta{\rm SNR} = 10\log_{10}\left( f )
\end{equation}
\includegraphics[width=0.01in]{blank}
\vspace{-10pt}
\caption{\textbf{Equations}.  Eq.~\ref{eq:one}
  is foo, while Eqs.~\ref{eq:two}
  and \ref{eq:three} are bar.}
\label{fig:eq}
\end{figure}

Bibliography

I spent the most time messing around with my bibliography. I wanted no line breaks between the items, the bibliography in a smaller font, only the first author listed and no titles in the items.

Achieving no line breaks requires two things. First, you have to find the *.bst file that LaTeX uses and copy it to your local directory under a different name. For example, I was using the "ieeetr" style file, so I copied ieeetr.bst from the location LaTeX uses to my local directory as myieeetr.bst. You then must modify this file, and comment out the "newline$" strings in the output item. So in my case, the item

FUNCTION {output.bibitem}
{ newline$
  "\bibitem{" write$
  cite$ write$
  "}" write$
  newline$
  ""
  before.all 'output.state :=
}

became (notice the %'s before the two newline$'s)

FUNCTION {output.bibitem}
{ %newline$
  "\bibitem{" write$
  cite$ write$
  "}" write$
  %newline$
  ""
  before.all 'output.state :=
}

Next, you have to add the following to the top of your .tex file before \begin{document} but after the usepackages

\usepackage{paralist}

\renewenvironment{thebibliography}[1]{\begin{inparaenum}[[1{]}]}{\end
{inparaenum}}

I have no idea what that command does, but my bibliography didn't have any line breaks. The bibliography stanza that comes after my main text ends is (and you should use whatever you named the local file in place of "myieeetr")

\medskip
\bibliographystyle{myieeetr}
\footnotesize{
\bibliography{myBibtexFileName}
}

To shorten the bibliography even further, I removed all "TITLE={...}," lines from my bibtex file, and replaced the non-first authors with the "others" string. This causes a bibliography item to look like

 [1] M. Smith, et al., Journal Name, vol. 99, pp. 123–123, Jan. 2000.

which is exactly what I wanted. If I was a manly user I'd modify the bst file to do this, but I'm only a lazypragmatic user. If you modify the ieeetr bst file to do this please send it to me!

Columns

I used one column for the text and "\twocolumn" before the figures to help with reducing dead-space.

Page Numbers

I used a "\pagestyle{empty}" at the start to remove page numbers.

Fonts

Adding a "\usepackage{times}" will help with using slighter smaller fonts.

Conclusions

I hope some of these notes will be helpful if you want to use LaTeX for conference papers. It was a bit challenging at first, but I'm very happy with the final results. Please let me know if you have any trouble getting things to work above; I'd be more than happy to send you my .tex files so you can see how things worked for me.

Superb Supercomputer Treemap

The BBC has a spectacular Supercomputer Treemap. It is definitely the classiest visualization I've seen all day. I suggest checking it out, along with the historical article on treemaps.

ACM Survery of Graphs

ACM has an interesting survey of graph types. I don't like most of them, *especially* graph types like the stacked graphs in 1B that introduce correlations into the visual representation of the data that aren't in the data. The splom in Figure 2C, however, is a very nice representation that I'd never seen before.

Greater than One?

Nothing increases my faith in science like seeing a highly cited article that experimentally measures a probability larger than one. Hmmmmmmmmm.

Game Review: Dwarf Fortress

I've been pretty hosed at work lately, so I apologize for the lack of recent posts. It is hard to write a blog when you've spent the past few weeks writing a paper (or three) at work. However, I just wanted to practice my non-technical writing and talk a bit about a Dwarf Fortress, a game that I thought I'd review.

The game has three modes: fortress, adventure and legend. All three modes center on different aspects of a single world - for the sake of brevity I'll be focusing solely on fortress mode, which is a mode similar to Sim City but with a fortress instead of a city. Adventure is a hack-and-slash mode similar to NetHack, and legend is an encyclopedic mode that is still in the works.

Two shots from legends mode show information a dragon that was slew in the year 118. If you like reading encyclopedias, you'll love legends mode!

The basic premise behind the game is all there in the title. You have dwarves, and you want to build a fortress in the side of a mountain (mountain optional). If you enjoyed playing Sim City, you'll probably enjoy Dwarf Fortress, though it has a *much* steeper learning curve. There isn't an end goal to the game, though as your fortress gains wealth and population your fortress will receive noble dwarves, such as a baron(ess) or a count(ess). The easiest way to create wealth is through workshops, though other resource-gathering is available. The population either grows from natural procreation or immigration.

The entrance to a fortress is shown at the left-hand side. There are two ballistas in the center-left defending the fortress entrance, along with two rooms straddling the entrance to allow cross-bow wielding dwarves to defend the entrance. On the right hand side of the screen is a bridge (red double-tildes with outline) crossing a brook (blue tildes). A dwarf is chilling on the bridge. The middle portion of the screen contains three catapults with ammo stock-piles to the left of them.

The first thing you'll probably notice is that the game has an ASCII tile-set. If you're not used to ASCII games like NetHack, you might have a very hard time adapting. There are other tile-sets that are a little easier on the eyes. I personally don't mind the graphics so much as the limited view of the world. Many modern games use zoom mechanics to deal with complexity, but it isn't possible with Dwarf Fortress.

The game can be split into roughly three categories: building/mining/economy, combat, and history. I'll tackle the subjects one at a time to review the game

History

Dwarf Fortress keeps track of almost everything. The legends screens above about the dragon? One of those is created for each one of your dwarves, tracking the life events of the dwarf. Every sentient or large creature that your dwarf kills is tracked. Your dwarves can be happy, sad, or go crazy if their spouse dies. Baby dwarves can go insane if their parent dies. This, too, will be recorded.

The history system can be a bit overwhelming at times, but at the same time it feels as though the game you're playing has an epic quality to it. You're never forced to look at the history system, which I think is a big plus. Did a dragon just arrive at your fortress and take out half your population before one of your champions brought it down? You won't feel so bad after you look at the legend screen and find out that once upon a time that dragon killed the elven king.

However, if that dragon takes out your fortress, it is gone. Dwarf fortress is a one-way game with no mulligans. You can only pause the game and quit. You can never save the game, make a mistake, and then reload the game. If a dwarf dies, they're dead. This may make it difficult for some players, and if you like do-overs then this game probably isn't for you.

Combat

The combat system allows dwarves to train, equip specific weapons and shields, and become injured or die. The detail found in the history system also exists in the combat system. Axe-dwarves or sword-dwarves will lob off heads, while marks-dwarves (cross-bow wielders) will kill enemies by puncturing vital organs. You can find out (if you wish) that your marks-dwarf killed that fox by puncturing its spleen before the fox bleed out. Your soldiers will break bones that will take months to heal while doctors attend to them.

Overall the combat system is a nice balance to the economy, which I'll talk about below. One disadvantage with the system is that fights tend to be very one-sided. If a dragon shows up, your dwarves are either prepared or they're dragon-meat. There isn't really a delay-the-dragon-while-dwarves train option. Goblins will occasionally siege your fortress, which (again) isn't a problem if you're prepared.

Control with the combat system is probably the weakest link in the game. You can lump dwarves together into a squad, but if the squad leader is sleeping when the goblins attack you have to re-arrange the squad. I think it takes too long to re-arrange the military to respond to the threat. Controlling your civilians is also difficult; control is very coarse, only allowing you to shut off whether civilians or soldiers should stay underground. Additionally, civilians complain so much when they're forced to stay inside that it makes playing the game difficult during an attack. All you see is "civilian complains about going outside because civilian wants to do X." I think this area needs a little bit of work before the game can shine.

(above) Dwarves train in the barracks on the top left.

(above) Examining the wounds of a dwarf. The yellow text signifies a broken bone.

(above) Some goblins (purple and white 'g' characters) ambush and kill a bone carver (dead blue dwarf).

(above) An axe-wielding soldier (blue dwarf) is about to "axe"-xact revenge on the goblins.

(above) The goblins didn't have a chance.

Resources for the Economy

I love building games like Sim City or Civilization. Dwarf Fortress has a great building system. You start by ordering dwarves to mine out an area of a mountain, creating a safe haven from the outside world. You then can create areas for stockpiles of resources, and workshops to turn the resources into either goods or other resources. A trade system exists with caravans to trade for goods. There is a complex assembly line for some goods - for example, to make steel armor, first you'll need steel bars. To get steel bars, a dwarf needs to process iron bars, a flux stone, and charcoal at a smelter. The charcoal needs to be produced at a wood furnace, and the flux stone needs to be mined from specific types of minerals. Charcoal is also used in other industries besides metal-smithing, such as glass-making, so to optimize the resource through-put you have to lay out your fortress stock-piles and workshops correctly.

Creating an effective economy is quite difficult. The distance to the surface of certain industries, such as the wood industry, needs to be balanced against fortress defense. Some structures, such as farms, have restrictions on what type of soil you can build them on top of. Your dwarves need to eat and sleep, but they'll have trouble sleeping near noisy workshops or eating near garbage.

When I first started playing the game I found myself thinking in 2-D...but over time, I realized that highly optimized fortresses needs to be built in 3-D. The extra dimension really adds a lot to the game if you love building assembly lines or working with resource management.

(above) Fields next to a river.

(above) Quarters with a bed and a door for about 100 dwarves. A set of nicer quarters with smoothed walls is in the middle

(above) A stock-pile of furniture on the left.

(above) A set of workshops spread across two rooms to the left of the hallway. There is a nearly empty stockpile of wood below one row of workshops (empty stockpiles are denoted by "=" signs).

(above) A dining room to the left of the hallway, with a garbage dump on the right. A row of workshops is below the garbage dump. The purple cloud is a miasma cloud coming from decomposing waste.

One of the most novel features of the game is the trap and pump system. You can pump water, and then use pressure plates to activate or deactivate flood-gates or pumps. If you want to build an auto-filling cistern or an underground irrigation system, you can set it up so you only have to pull a single lever to get your pumps and flood-gates to do what you want. Are you having trouble with goblins attacking your fortress? Place a puppy some food outside as bait and then drown the goblins when they step on a pressure plate. If you prefer to catch the goblin, use the plate to slam doors shut or just use an old-fashioned cage trap. You could build an arena and then have goblins fight to the death! Or maybe start a goblin breeding program and use the baby goblins as a source of totems.

The building system, I think, is the strongest feature of Dwarf Fortress. The possibilities are endless - whether you want to build an arena or figure out how to build a computer, the open-ended system is one of if not the best I've seen in a game.

Overall

Dwarf Fortress is an old school ASCII game with a lot of promise. If you can look past some of the rough edges with the combat system and incomplete features (the game is still an alpha), the game plays out like Sim City meets tinker toys. Much like Sim City, there is some dead-time while you wait for things to happen, but there is a great reward for building that auto-filling cistern. Overall if you liked NetHack and Sim City I think you'll love this game; but beware the high learning curve almost mandates using a guide to help you out with the first few steps.

Tip About Airline Flights

On my recent trip from Amsterdam to Boston to Iowa to Amsterdam, I needed to move my last set of flights forward a day because of an important meeting. I called up American Airlines, and because my Boston to Iowa trip had been delayed, AA didn't charge any money to change my flight to Amsterdam.

If you ever need to reschedule flights, AA will be very accommodating if you've had previous delays. This is probably true for other airlines. I hope someone finds this tip as useful as I did.

Grocery Stores in Iowa

How can any store carry hummus but not pita bread! They even had the extra garlic type of hummus.

Well, I guess that is Iowa for you. Sigh.

A Little Bit of Dancing

Congratulations to my big sister, who was married this weekend! There were some other photos that were shown at the wedding, but this one was my favorite; I think it sums up a lot of the past.

California Iowa Here I Come

I'm on vacation for the next two weeks - woohoo! Looking forward to seeing you all.

Sorry about Anonymous Commenting Being Turned Off

A loyal reader has informed me that anonymous commenting was turned off. My apologies, I've switched it back on.