Skip to main content
USENIX
  • Conferences
  • Students
Sign in
  • Overview
  • Workshop Program
  • Call for Papers

connect with us


  •  Twitter
  •  Facebook
  •  LinkedIn
  •  Google+
  •  YouTube

twitter

Tweets by @usenix

usenix conference policies

  • Event Code of Conduct
  • Conference Network Policy
  • Statement on Environmental Responsibility Policy

You are here

Home » Collecting and Analyzing Provenance on Interactive Notebooks: When IPython Meets noWorkflow
Tweet

connect with us

Collecting and Analyzing Provenance on Interactive Notebooks: When IPython Meets noWorkflow

Authors: 

João Felipe Nicolaci Pimentel, Vanessa Braganholo, and Leonardo Murta, Universidade Federal Fluminense; Juliana Freire, New York University

Abstract: 

Interactive notebooks help users explore code, run simulations, visualize results, and share them with other people. While these notebooks have been widely adopted in teaching as well as by scientists and data scientists that perform exploratory analyses, their provenance support is limited to the visualization of some intermediate results and code sharing. Once a user arrives at a result, it is hard, and sometimes impossible, to retrace the steps that led to the result, since they do not collect the provenance for intermediate resuls or of the environment. As a result, users must fulfill this gap using external tools such as workflow management systems. To overcome this limitation, we propose a new approach to capture provenance from notebooks. We build upon noWorkflow, a system that systematically collects provenance for Python scripts. By integrating noWorkflow and notebooks, provenance is automatically and transparently captured, allowing users to focus on their exploratory tasks within the notebook. In addition, they are able to analyze provenance information within the notebook, to both reason about and debug their work, using visualizations, SQL queries, Prolog queries, and Python code.

João Felipe Nicolaci Pimentel, Universidade Federal Fluminense

Vanessa Braganholo, Universidade Federal Fluminense

Leonardo Murta, Universidade Federal Fluminense

Juliana Freire, New York University

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {192113,
author = {Joao Felipe Nicolaci Pimentel and Vanessa Braganholo and Leonardo Murta and Juliana Freire},
title = {Collecting and Analyzing Provenance on Interactive Notebooks: When IPython Meets noWorkflow},
booktitle = {7th {USENIX} Workshop on the Theory and Practice of Provenance (TaPP 15)},
year = {2015},
address = {Edinburgh, Scotland},
url = {https://www.usenix.org/conference/tapp15/workshop-program/presentation/pimentel},
publisher = {{USENIX} Association},
month = jul,
}
Download
Pimentel PDF
  • Log in or    Register to post comments

© USENIX

  • Privacy Policy
  • Conference Policies
  • Contact Us