Graphical Notebooks

Life beyond the computational essay


Alex Morley


Mozilla Fellow - Neuroscientist

Let's create our own adventure

Did anyone do any create your own adventure books as a kid? Or maybe you watched bandersnatch on Netflix at the start of the year. Whatever, we're gonna do a democratic one right now.

It's a sunny day in Portland, Oregon. You've just left home, and are walking to give a presentation about some work you are super excited about. The sidewalk you are walking on is tiled. Do you:

a. ignore the tiles and continue walking

b. adjust your stride so that you don't step on the edges between tiles

As you carry on walking some dude looking at his phone almost walks into, luckily you sidestep and avoid having his coffee poured all over you. The coffee smells pretty good though, and Upper Left Roasters is just round the corner. Do you:

a. go grab a coffee to give you energy for your talk

b. carry on walking so that you don't end up late

You pick up the pace and walk briskly to the coffee shop. It's been a while since you got a coffee from here and the coffee at work just doesn't cut it. As you walk in to the shop you see a friend from high-school.

"Grace! How are you?!"

It turns out Grace is sad because she's just about to move back to Chicago and the home she found for her cat Tommy can't take her anymore as they've found out their kid has allergies.

You adopt a cat!

You carry on walking in the sun. The air is fresh but the sun keeps you warm even without a sweater. You get to the room where you'll be presenting 20 minutes early but there's someone already waiting here for you. Shit it's Michelle Obama.

"Hey Jane, I don't have time to stay for your talk but I just had a meeting next door and saw the advertisement for the talk. It sounds super interesting, would you be interesting in collaborating to build an organisation around this?"

Fuck!

You didn't notice the dude looking at his phone who walks into and spills his coffee on your shirt. There's no way you can give your talk looking like this. Do you:

a. Have a look in the shop round the corner. Your favourite jumper is from there but it's mostly second-hand so they don't always have your size.

b. Run back home. You should be able to make it there and back in time for your talk.

You pick up the pace and walk quickly to "Hats 'n Cats", your favourite clothes store. Lucky it's open! You have a quick look around and then stop: no way! they have the perfect shirt and it goes perfectly with this jacket.

You run home. It feels good to get rid of some of this nervous energy even if it's a little warm. You sidestep and dart to keep your feet on the centre of the sidewalk tiles, even when running. You get home with more than 25 minutes before your talk and ... crap you had left your notes on the table! Lucky you came back anyway.

Essay or choose-your-own-adventure?

Sometimes we just want to tell a nice story.

Sometimes we want to share the whole process.

Jupyter notebooks are really great at doing the first. But sometimes they can be a little unweildy. But there are a whole bunch of anti-patterns, or workaround behaviours that I've observed that stem from the disconnect between natural process of data-analysis, and the format in which we express that analysis on the page. I'm going to pick a few of these, and then present some potential design solutions.

Observations:

analysis-copy-1.ipynb, analysis-copy-2.ipynb, analys...

Ctl-C, Ctl-V

All of the ####

Explanations:

Not sure if I'll need it later

Don't want to lose work

Too frequent for version control

...

Observations:

Hidden State Bugs

Many shared cells between notebooks

Scrolling a lot

Notebook Maps ... (stickies, paper, whiteboard etc.)

Explanations / Requirements:

Explore multiple overlapping avenues for analysing a dataset

Controlled sharing of state!

...

Mutliple kernels

Object "workspace" serialization

Process state capture

Observations:

Difficulty onboarding new users to a processing pipeline

Debugging/Reporting stages mixed in with actual processing

Complex static diagrams approximating the code

Explanation:

No visual/familiar way to edit pipeline

Implementation: Pipeline Kernels

Make

Common Workflow Language

Apache Kafka

### Story It's a sunny day in Portland, Oregon. You've just left home, and are walking to give a presentation about some work you are super excited about. The sidewalk you are walking on is tiled. Do you: a. ignore the tiles and continue walking b. adjust your stride so that you don't step on the edges between tiles --- As you carry on walking some dude looking at his phone almost walks into, luckily you sidestep and avoid having his coffee poured all over you. The coffee smells pretty good though, and Upper Left Roasters is just round the corner. Do you: a. go grab a coffee to give you energy for your talk b. carry on walking so that you don't end up late --- You pick up the pace and walk briskly to the coffee shop. It's been a while since you got a coffee from here and the coffee at work just doesn't cut it. As you walk in to the shop you see a friend from high-school. "Grace! How are you?!" It turns out Grace is sad because she's just about to move back to Chicago and the home she found for her cat Tommy can't take her anymore as they've found out their kid has allergies. You adopt a cat!