Curiosity

Shell notebooks

A Shell notebook is a tab of the admin Shell that runs code in cells and keeps its variables from one cell to the next. It comes in two languages: C#, in the same scope a Shell script runs in, and Python, with that scope available as a module. This page covers what a notebook is, what it can reach, and its limits. The language pages show the code.

Notebooks are for exploring: run a query, look at the result, refine it, chart it, keep going. A script you want to run on a schedule or call from outside belongs in a custom endpoint or a scheduled task.

Open a notebook

Open Shell in the admin console, select New tab, and choose what the tab is for:

Tab type What runs State between runs
Script (C#) One C# script, start to finish, in the Shell scope. None. Each run starts clean.
Notebook (C#) C# cells in the Shell scope. Variables, methods and types declared in one cell are available in the next.
Notebook (Python) Python cells in an in-process interpreter, with the Shell scope as the shell module. One namespace for all cells.

Only system administrators can open a notebook. Each tab is its own notebook with its own variables, so one notebook cannot see another's.

Work in a notebook

Run the selected cell with Shift + Enter. The toolbar above the cells has one icon button per action; hover a button to see its name.

Button Action
Run Runs the selected cell.
Run all Runs every cell with code, top to bottom, and stops at the first one that fails.
Stop Interrupts the cell that is running. See stop a running cell.
Add cell Adds an empty cell at the end.
Restart Ends the kernel and forgets every variable. The cells keep their code.
Variables Lists the variables the cells defined, with their types.
Clear outputs Removes the output under every cell.

Each cell also has a run button, which turns into a stop button while that cell runs, and a delete button. The number beside a cell is its execution count: [3] means it was the third cell run in this kernel, [*] means it is running, and [ ] means it has not run.

The code of the cells is kept in your browser's local storage, so a reload brings the notebook back. Outputs are not kept: run the cells again to see them.

What a cell shows

A cell shows what it printed, then the value of its last expression, then an error if it failed, in the order they happened. Ending the last line of a Python cell with ; runs it without showing the value.

Output Shown as
Text and print output Text. Standard error is red.
A query, or the result of one The viewer the Shell's Results pane uses for a returned query, with its render mode selector.
A chart The chart. See charts and tables.
HTML, SVG In a frame with scripts, forms and navigation switched off. A pandas DataFrame displays as an HTML table.
Markdown Rendered Markdown.
PNG, JPEG The image.
Anything else Its text form.

The kernel behind a notebook

A notebook runs on a kernel, a session on the server that holds its variables. A kernel starts when you run the first cell, and:

  • One cell runs at a time in a kernel. Running a second cell while one is running shows a message and does nothing.
  • Reloading the page reconnects to the same kernel, so the variables are still there.
  • A kernel ends when you close its tab, when you select Restart, or after 30 minutes without a request. The next run starts a new kernel, and a toast says the variables are gone.
  • Each administrator can have 8 kernels open. A ninth returns Too many notebooks are open. Close one first.

Stop a running cell

Stop cancels the CancellationToken of the scope for the cell that is running. Each cell gets a token of its own, so a stop affects that cell only: the kernel stays up and every variable is kept.

  • In C#, queries, endpoint calls and anything else that watches CancellationToken stop, and the cell fails with The cell was interrupted. Code that never looks at the token runs to its end. See the C# page.
  • In Python, the interpreter stops at its next instruction boundary and the cell fails with KeyboardInterrupt. No except in the cell can catch it.

A query a cell kept in a variable keeps the token it was created with. After a stop, create the query again in the next cell.

What a notebook can reach

A notebook has the reach of a Shell script, and its code is logged the same way.

  • The graph. Both languages run as the administrator who opened the notebook, in the Shell scope: Q(), Graph, CurrentUser, ChatAI and the rest of the shell scope. With the Sandbox toggle on when the notebook opens, they run in the sandboxed shell scope, which reads the graph and cannot write to it.
  • Audit. The code of every cell that runs is written to the audit log as a Shell run is, and to the server log.
  • No host access from Python. The interpreter has no access to the server's files, the network or processes. It has a scratch file system in memory, private to the notebook and gone when the kernel ends. Graph.Internals is not reachable from Python.
  • C# is C#. A C# cell can do what any Shell script can. The notebook adds no restriction and no permission.

Limits

Limit Value
Notebooks open per administrator 8
Idle time before a kernel ends 30 minutes
Python instructions per cell 20 million. A loop that runs longer fails with RuntimeError: instruction limit exceeded.
Rows in a chart 50,000
Rows when turning a query into a table 100,000
Cells running at once in a kernel 1

Known gaps:

  • The C# editor in a notebook has completion and hover for the scope, but no live compile errors, because the checker does not know the variables of earlier cells. Compile errors appear when the cell runs, with their line and column.
  • C# completion does not offer variables declared in earlier cells.
  • A Python cell that ends in a call to a function you wrote does not complete the result of that call. Completion never runs your code.
  • Methods that take a delegate or a type argument, such as RunEndpointAsync<T>, cannot be called from Python.

A Python notebook runs on an interpreter written in C#, so loops in a cell are much slower than in a C# notebook, while a cell that is mostly one query is close. How Python notebooks run, and how fast shows the measurements.

Next steps

  • C# notebooks shows variables across cells, querying, stopping and charting.
  • Python notebooks shows the shell module, DataFrames, pandas and the naming rules.
  • Charts and tables turns query results into charts in both languages.
  • The reference lists every Python name, the flint module, the C# Plot API and the notebook endpoints.