Plot or Not? Voting Results

Screenshot from plotornot

Screenshot from plotornot

During the SciPy conference last June, Adrian Price-Whelan and I got to commiserating about the ugly default styles of Matplotlib plots — something that also came up at the Matplotlib town-hall meeting. Because Python’s main plotting library was designed as an alternative to Matlab, it inherited much of the appearance and API from that language. This was originally an asset for Matplotlib, as it provided an obvious path for users to migrate away from Matlab. Unfortunately, as Matplotlib has matured, it’s core visual and programmatic design has calcified. People who want to create nice plots in Python are forced to write lots of tweaking code.

Users who push for changes to the default appearance of plots usually face 3 challenges:

– While most agree that Matplotlib would benefit from a better style, there is less consensus on what a replacement style should look like.

– Some are skeptical about the subjectivity of plot aesthetics, and think that “improvements” to plot styles are really just pleasing some users at the cost of others.

– Matplotlib developers want ot avoid changes which might “break” the appearance of legacy user code running in pipelines

As Adrian and I discussed this, we wondered what it would take to integrate substantial stylistic changes into Matplotlib itself. We realized there’s very little data on what kind of Matplotlib plots people actually like. So we decided to collect some. During the SciPy sprints, we put together Plot or Not?, which randomly showed visitors the same Matplotlib plot rendered with two different styles, and asked which one they preferred. People liked it: the site crashed (I apparently don’t know how to make websites with actual traffic), we acquired about 14,000 votes, somebody suggested the name was misogynous, we triggered a good discussion on the Matplotlib developer mailing list, and we promised to share the voting results soon. Then I remembered I had to finish my thesis, and the data sat on a server somewhere for 6 months.

As luck would have it, I found some time to dig into the votes this weekend. You can explore the results at this page, which shows a scatter plot of each Plot or Not image as a function of the fraction of votes it received (X axis) and the margin of victory/defeat (Y axis). Clicking on a point will show you the voting breakdown for a given face-off.

The dataset, it turns out, has a lot of interesting information about what kinds of plots people like. You should explore for yourself, but here are some of the biggest themes I noticed:

People largely share the same aesthetic preferences

Yes aesthetics have a large subjective component. However most people agreed on which plots they preferred. This argues that there are stylistic changes one could make to Matplotlib that would be a net improvement, despite subjectivity.

Legibility is the most important factor

If you look at the heaviest favorites, many of them are comparisons between a plot with easily-seen lines and one whose lines are too thin, too transparent, or too light.

Cam Davidson-Pilon’s Style is Very Good

For some of the plots we generated, we used the settings Cam Davidson-Pilon used in his online book.These were consistently selected as the favorite, and often by large victories like 5:1 or more.

We also used the style from Huy Nguyen’s blog post, which emulates GGPlot — it’s very similar, though it uses a thinner font and linewidth. People slightly preferred Cam’s style in head-to-head comparisons — probably because the lines are easier to see.

People like the dark Color Brewer colors (but not the pastel ones)

Many of the plots in plotornot used colors from colorbrewer2.org. People liked line plots and histograms that used the Set1 and Dark2 color palettes. Likewise, people often preferred filled contour plots that used the divergent Color Brewer palettes.

However, people did not like plots drawn with pastel Color Brewer tables (Pastel, Accent, Paired2, Paired3). These are both harder to see, and feel a bit… “Easter-y” (this is a highly scientific adjective). Unpopular colors for contour plots included Accent, Prism, HSV, and gist_stern. All of these palettes cycle through several hues. It is hard to encode scale with Hue, and people preferred palettes restricted to one or two hues. In fairness, some of these multihue palettes would have looked better on images that encode more than ~5 values at a time. Still, the advice from visualization experts seems to be to stick to one- or two-hue colormaps. The latter are best suited in cases where you want to call attention to outliers with both large and small values.

The default Matplotlib colors are almost never preferred

Unlike the Color Brewer colors — which are designed for legibility and coherence — the default Matplotlib color set is pretty arbitrary (blue, green, red, cyan, magenta, yellow, black). These colors don’t work well together, and it shows in the votes. In the few instances where a matplotlib default was preferred, the other plot usually had hard-to-see lines.

An easy improvement

There are a lot of ways one could consider changing styles in matplotlib. The votes from Plot Or Not? suggest a few obvious improvements:

– Use the Set1 or Dark2 Color Brewer palettes for the default line style

– Use a single-hue colormap like ‘gray’ for the default color map.

– Increase the default linewidth from 1 to 2

While the Matplotlib devs are still resistant to changing any defaults, there are some improvements that you will start to see in Matplotlib v1.4. This includes a “style.use” function which will let you easily select style sheets by name or filepath/url. For example, to use the style changes advocated for in this blog post, you could write

from matplotlib.style import use
use('http://plotornot.chrisbeaumont.org/matplotlibrc')

My hope is that Matplotlib will start to build some nice stylesheets that ship with the library. Eventually, I would also love to see a new option for my matplotlibrc file that specifies ‘default_style: latest’ — this would indicate that I am “opting-in” to whatever the Matplotlib developers deem to be the best default style. This style could then incrementally improve with each release, without breaking any legacy code.

In the meantime, the 6 months since SciPy have seen a lot of progress on viz libraries which build on top of (seabornprettyplotlib, a ggplot clone, mpld3, glue) or offer alternatives to (vincent, bokeh) Matplotlib. I’m excited about all of these projects, but hope also that Matplotlib is able to keep evolving to stay modern (I haven’t talked at all about Matplotlib’s API, but I would love to see that improve as well). Matplotlib has solved a lot of problems and remains the most mature library for plotting in Python by far. Even incremental improvements to Matplotlib can have a big effect on the Python community.


The Aspects of Astronomy in the Cloud That Scare Me

I  spent the last two days in a very interesting discussion group about visualization challenges for ALMA. ALMA is arguably the first observatory where the data products will routinely lie in “big data” territory — that is, the Gigabyte-Terabyte range where data sets can’t easily be analyzed on a single machine. We’ve created observational datasets this large before, but they have arguably been niche products that only a few researchers use in their entirety (large swaths of the entire 2MASS or Sloan surveys, for example). Many, many people who use ALMA data will have to contend with data sizes >> RAM. The community needs to come up with solutions for people to work with these data products.

The big theme at this discussion group was moving visualization and analysis to the cloud, where more numerous and powerful computers crunch through mammoth files, and astronomers interact with this resource through some kind of web service. We spent a lot of time looking at a nice data viewer and infrastructure developed in Canada that is great for browsing through 100GB (and larger) image cubes.  Yet I find myself uneasy about this move to the cloud. I seemed to be in the minority within the group, as most others embraced or accepted this methodology as the inevitable future of data interaction in astronomy (I may or may not have been called a dinosaur — admittedly, I was being a bit obnoxious about my point!).

I get that cloud computing is unavoidable at some level — most astronomers do not have nearly enough computational resources or knowledge to tackle Terabyte image cubes, and we will need to rely on a centralized infrastructure for our big data needs. Centralized resources are also great for community science, where lots of people need to work on the same data. But in an attempt to defend (or at least define) my dinosaur attitudes, here are the issues that I think astronomy cloud computing needs to address:

Scope of access: How often and to what extent will an observer have access to cloud resources? Will she be able to visualize data whenever she wants? Will she be able to run arbitrary computation? How much of a lag will there be between requests and results? Many of us are used to a tight feedback cycle when visualizing, analyzing and interpreting data. Is it a priority to preserve this workflow? Is that technologically and financially feasible?

Style of access: How many ways will we be able to interact with data? What restrictions will be placed on the computation and visualizations we undertake? Will we be able to download smaller sections of the data product for exploration offline? Will this API be in a convenient form (python library, RESTful URL, SQL) or some more awkward solution (custom VO protocol, cluttered web form)? What will the balance be between GUI and programmatic access? How well will each be designed and supported (personally, I can tolerate a poor GUI interface much more than a bad programming library)?

Bottlenecks for single machines. Underlying all of this is is the assumption that it is impossible to work with ALMA data on local machines. I think this is overhyped in some aspects. Storing even a Terabyte of data is trivial (1 Tb hard drives are $100, compared to $2000 per year to store 1 TB on Amazon’s cloud, to say nothing of computation). While churning through all of this data is certainly a many-hour task with a single disk, many operations relevant for visualization, exploration, and simple analysis are trivial (extracting profiles, slices, and postage stamps on a properly indexed data cube is very cheap, and gives you a lot of power to understand data and develop analysis plans). Should we really fully abandon this workflow that almost all astronomers currently use? Is it worth developing new software to help interact with local data more easily?

By no means are these issues insurmountable, and I was probably sweating the details too much for the high-level discussion at the meeting. But the details do matter, and the Astronomical community has had a mixed track record with creating interfaces to remote data products (new visualization clients are getting pretty good, but services for analysis or data retrieval are still pretty cumbersome). My reaction to most of these clumsy products has been to avoid them, because it has been possible to fetch and analyze the data myself. Once we lose that ability, we will all become very dependent on external services. At that point, the details of remote data interfaces may become the new bottleneck for discovery.

RAWRRRR (dinosaur noises)


Critiquing the Visual.ly Divorce Post

Update: Paul Van Slembrouck, the designer of this graphic, has responded to the critique. Be sure to read his comments below!

I am a teaching fellow for a class at Harvard called “The Art of Numbers,” which teaches principles of data presentation to undergraduates from all concentrations. For a recent midterm, students were asked to analyze this graphic from Visual.ly:

Distribution of education levels for women who divorced in 2008

For valentine’s day Visual.ly posted a series of visualizations of divorce statistics in the U.S.. Several aspects about this graph bothered me, and I thought it would make for a good exam question.

Read the rest of this entry »