Intro

After my first terminal rendering engine, I wanted something more. Stumbling upon
notcurses, A cross platform framework slash library for developing modern TUIs on
modern terminal emulators.

Before I found notcurses, I was messing around with SDL and it’s library functions
to expose a framebuffer to draw to. Wanting to do the same with notcurses, I went
straight to the docs. They were okay but I was in the dark about basically
everything, so I dropped the project.

Fast forward to the future, after searching through source code of the examples provided
by the notcurses developers, I figured out how to blit a framebuffer to the screen.
Where from now? Following from my GLSL shader raymarching project in V (l1mey112/raymarching-v)
using sokol, a high level OpenGL library, I sought out to render at least something
to the terminal display through notcurses.

You Have To See It To Believe It

Take a look at a recording of the first minute of the notcurses demo.

Now take a look at that same recording of the 60 fps video being played straight back to the terminal.

Fucking impressive right???

That tool used to play back video to the terminal was called ‘ncplayer’. A fully fledged
media player utilising ffmpeg as a backend and notcurses as the rendering frontend. The resolution
isn’t great, but it makes up for that in extremely high redraw rate being able to dish out frames
incredibly quickly.

Modern terminal emulators are fast, crazy fast. Notcurses seeks to bring the best out of them.

(P.S. I use the alacritty terminal emulator. Text rendering is GPU accelerated!)

It keeps rendering fast by assuming that whatever was placed on the screen last frame, stayed there.
It can then go over and only edit pixels which colour values have changed. It may not sound like much,
just a minor optimistation, but you would be wrong. Just think about how much overhead is created redrawing the
entire frame, each frame. Video compression relies on the fact that pixels tend to keep their values between
frames, imagine how much larger a video would be if it stored the entire frame instead of just interpolations
between a couple ‘I-Frames?’

Anyway, Here Is How To Use It

I won’t go over everything, just what I use. That-is what-the docs-is-for.

First, initialise notcurses and get a pointer to the standard plane. The standard plane
is always present and is the same size as the screen.

#include <notcurses/notcurses.h>

struct notcurses *nc = notcurses_init(NULL, stdout); 
  // the notcurses context
struct ncplane *pl = notcurses_stdplane(ncr->nc);
  // the standard drawing plane, always the size of the 
  // terminal window

ncblitter_e nb = NCBLIT_1x1;
  // NCBLIT_1x1      space, compatible with ASCII
  // NCBLIT_2x1      halves + 1x1 (space) ▄▀
  // NCBLIT_2x2      quadrants + 2x1 ▗▐ ▖▀▟▌▙
  // NCBLIT_3x2      sextants (*NOT* 2x2) 🬀🬁🬂🬃🬄🬅🬆🬇🬈🬉🬊🬋🬌🬍🬎🬏....
  // NCBLIT_4x1      four vertical levels █▆▄▂
  // NCBLIT_8x1      eight vertical levels █▇▆▅▄▃▂▁
  // NCBLIT_BRAILLE  4 rows, 2 cols (braille) ⡀⡄⡆⡇⢀⣀⣄⣆⣇⢠⣠⣤⣦⣧⢰⣰⣴⣶⣷⢸⣸⣼⣾⣿
  // NCBLIT_PIXEL    pixel graphics (if supported)

Notcurses supports using different characters to represent
each pixel, with each blitter type coming with it’s own
pixel aspect ratio and the like. Each blitter format changes the amount of
pixels that fill the terminal. The amount of rows and
columns on the terminal window do not necessarily
map to how many pixels you need inside a compatible framebuffer.

Below is the blitter section of the demo on a terminal emulator called
‘kitty’. Unlike alacritty, this terminal has support for images with it’s
own image protocol
natively supported by Notcurses instead of the legacy Sixel format.

I recompiled the demo with extra “demo_nanosleeps” in the places it was needed.

void blitter_real_dims(ncblitter_e nb, 
                       uint32_t *fb_r_x, 
                       uint32_t *fb_r_y)
{
    switch (nb)
    {
    case NCBLIT_1x1:
        break;
    case NCBLIT_2x1:
        *fb_r_y *= 2;
        break;
    case NCBLIT_2x2:
        *fb_r_x *= 2;
        *fb_r_y *= 2;
        break;
    case NCBLIT_3x2:
        *fb_r_x *= 2;
        *fb_r_y *= 3;
        break;
    case NCBLIT_BRAILLE:
        *fb_r_x *= 2;
        *fb_r_y *= 4;
        break;
    case NCBLIT_PIXEL:
    case NCBLIT_DEFAULT:
    case NCBLIT_4x1:
    case NCBLIT_8x1:
        break;
    }
}

With all everything finally set up, use this code to allocate a framebuffer to write to.

uint32_t fb_x, fb_y;                     // variables to store the rows and cols 
                                         // of the terminal window
ncplane_dim_yx(pl, &fb_y, &fb_x);        // get rows and cols from the std plane

uint32_t fb_r_x = fb_x, fb_r_y = fb_y;   // variables to store the size in pixels
                                         // for the framebuffer
blitter_real_dims(nb, &fb_r_x, &fb_r_y); // get the real framebuffer size

size_t fb_size = fb_r_x * fb_r_y * sizeof(uint32_t);
uint32_t *fb = malloc(fb_size);          
                                         // calculate the size and allocate a 
                                         // framebuffer

Rendering it is just as simple.

const struct ncvisual_options opts = {.n = pl,
                                      .scaling = NCSCALE_NONE,
                                      .leny = fb_r_y,
                                      .lenx = fb_r_x,
                                      .blitter = nb};

uint32_t fb_r_xl = ncr->fb_r_x * sizeof(uint32_t);
                                         // the size of a 'stride' of pixels. 
                                         // essentially how big in bytes is 
                                         // one horizontal scanline

ncblit_rgba(fb, fb_r_xl, &opts);         // 'blit' to the std plane
notcurses_render(nc);                    // render all planes to the screen

The Final API

Here is what an example render loop would bake down to in the version
of the project without OpenGL.

Want to use that version for your own? It’s in the first commit right here.

typedef struct
{
    struct notcurses *nc;
    struct ncplane *pl;
    ncblitter_e nb;

    uint32_t *fb, fb_r_x, fb_r_xl, fb_r_y, fb_x, fb_y;
} NCRenderer;

NCRenderer *ncr_init(ncblitter_e nb);
void ncr_fullscreen(NCRenderer *ncr);
void ncr_blit(NCRenderer *ncr);

static inline size_t ncr_sizeof_fb(NCRenderer *ncr);
static inline vec2_t ncr_aspect(NCRenderer *ncr);

int main(void) {
    NCRenderer *ncr = ncr_init(NCBLIT_1x1);

    while(true)
    {
        ncr_fullscreen(ncr);           // handle screen resizes and
                                       // allocate new fb if needed

                                       // god tier for loop incoming
        size_t fb_s = ncr_sizeof_fb(ncr) / sizeof(uint32_t);
                                       // 4 byte wide memcpy
                                       // over the whole fb     
        for (uint32_t *p = ncr->fb, n = 0;
             n < fb_s;
             n++, *p++ = 0xFF131313);

        size_t fb_stride = ncr->fb_r_xl / sizeof(uint32_t);
        size_t idx = fb_stride * (ncr->fb_r_y / 2)
                     + (ncr->fb_r_x / 2)
                     - 1;
                                       // y * x_stride + x

        ncr->fb[idx + 0] = 0xff0000FF; // rgba(1, 0, 0, 1)
        ncr->fb[idx + 1] = 0xff00FF00; // rgba(0, 1, 0, 1)
        ncr->fb[idx + 2] = 0xffFF0000; // rgba(0, 0, 1, 1)

        ncr_blit(ncr);                 // render fb to screen
    }
}

This is it, handling resizes and all.

It’s time for OpenGL.

The OpenGL ‘context’.

I don’t know much about OpenGL, being my first proper OpenGL project figuring this all
out took ages.

OpenGL requires a context to function. Generally this is handled by the operating
system on window creation by interfacing with whatever display server running.

I don’t need a GUI window, the terminal is all I need to render to.

OpenGL was never designed for rendering to anything other than a window. It is so tightly
integrated with an operating system’s windows along with it’s display drivers.

All I wanted to do was write a screen shader for performing rendering techniques like
raymarching and raytracing, which are both completely separated from what modern GPUs
are optimised for. The (rasterising) rendering pipeline.

General computation on the GPU outside of the rendering pipeline is more suited to OpenCL,
which doesn’t require any kind of window and is OpenGL’s counterpart for GPU based
parallel computation. I mean it seemed pretty good, until I realised OpenCL was
just C with nonstandard extensions for the GPU. Not good for graphics, I wanted
GLSL and all of it’s quality of life features for working with vectors and matricies.

Compute shaders were introduced into the OpenGL specification for this, skipping the
pipeline to render straight to a texture. But what’s the point of a compute shader if
you still need a window?

I compromised instead.

How did I do it?

First, I used the GLFW library to create a hidden window. (GLFW Docs)

#include <GL/glew.h>
#include <GLFW/glfw3.h>

glfwInit();                            // init GLFW
glfwWindowHint(GLFW_VISIBLE, 0);       // force window to be hidden

GLFWwindow *offscreen_ctx = glfwCreateWindow(640, 480, "", NULL, NULL);
                                       // create window, initial
                                       // dimensions mean nothing

glfwMakeContextCurrent(offscreen_ctx); // this is my new context
glewInit();                            // init OpenGL functionality

Since all we use the hidden window for is for an opengl context, we cannot render to it.
Nothing will be shown. You need to create a secondary framebuffer to replace the existing
default one. Every framebuffer object needs at least one texture for colour to render to,
create that also.

LearnOpenGL was an incredible help figuring all this out.

GLuint fbo;                             // most objects in OpenGL are stored as an
glGenFramebuffers(1, &fbo);             // unsigned 32 bit integer

glBindFramebuffer(GL_FRAMEBUFFER, fbo); // bind the framebuffer to the window, this 
                                        // will be the new render target

GLuint fbtex;                           // OpenGL texture boilerplate incoming
glGenTextures(1, &fbtex);
glBindTexture(GL_TEXTURE_2D, fbtex);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 640, 480, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, fbtex, 0);

assert(glCheckNamedFramebufferStatus(fbo, GL_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE);
                                        // checking if everything went smoothly

Again, the 640 by 480 size does not matter. Textures in OpenGL are dynamic, another call to
‘glTexImage2D’ with the correct bounds will automatically fix everything up.

Now that the display is using a different framebuffer, one that we can peek into using it’s
texture, simply copying it straight out to the notcurses framebuffer doesn’t take much more
code. That’s the complete setup done.

The Render Loop

Inbetween the notcurses calls start rendering absolutely anything using OpenGL. Anything.

I recomend the OpenGL’s version of a ‘hello world’, the hello triangle.

But if this is your first project using OpenGL, why are you here?

while (true)
{
    ncr_fullscreen(ncr);

    glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
    glClear(GL_COLOR_BUFFER_BIT);

    // glDrawWhatever();

    ncr_opengl_blit(ncr);
    ncr_blit(ncr);
}

The function below is where all the magic happens, taking the image data from the
framebuffer’s texture and writing it to the framebuffer.

void ncr_opengl_blit(NCRenderer *ncr)
{
//  glPixelStorei(GL_PACK_ALIGNMENT, 1); // may be needed
    glActiveTexture(GL_TEXTURE0);

    glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_UNSIGNED_BYTE, ncr->fb);
//  glReadPixels(0, 0, ncr->fb_r_x, ncr->fb_r_y, GL_RGBA, GL_UNSIGNED_BYTE, ncr->fb);
//  functionally equivalent
}

I mean, that’s literally it. The real hard part was just learning OpenGL the unconventional way… and figuring out that above function.

Down below is the hello triangle, I followed a tutorial on open.gl instead of the one on LearnOpenGL
because it shows you how to implement vertex attributes. Vertex attributes were needed so I could get UV coordinates on the screen.

Wait, that doesn’t look right. The triangle is upside down!

I didn’t do anything wrong, this is just how OpenGL works internally. Framebuffers are always flipped on the vertical axis and
appear the right way up when rendered to a window, we aren’t using one.

How this is solved is up to you.

Flipping each individual pixel from top to bottom is incredibly expensive, do NOT do this. Using OpenGL proper with
camera matricies? Just invert the vertical component with zero cost to rendering. Just using screenspace rendering
with the fragment shader and a screen wide quad/triangle? You can flip the polygon or give it flipped UVs, your call.

Hold on, a screen space shader using a triangle instead of a quad?

It’s a nice trick from the Godot Engine documentation, here.
While it is also a small rendering optimistation, it saves me from dealing with index buffers. Extra boilerplate I don’t need.

Here is the flipped triangle and it’s UV vertex attributes.

const vertex_t vertices[] = {
    { 3.0,  1.0, 0.5, 2.0, 0.0},
    {-1.0, -3.0, 0.5, 0.0, 2.0},
    {-1.0,  1.0, 0.5, 0.0, 0.0},
};

typedef struct
{
    float x, y, z;
    float u, v;
} vertex_t;

The End

Without notcurses, writing pixels to the screen would be the bottleneck.

With notcurses, the GPU is the bottleneck!

To demo it all, as tradition, I wrote a fragment shader to compute the mandelbrot set.

Scroll the mouse to zoom in and out, arrow keys to move around.

You see the pixels glitching out there at the end as I zoom far in? Yeah, GPUs aren’t good with floating point precision. Any serious
fractal render does calculations on the CPU with SIMD instructions to parallelise it all.

int a;
int b[2];
glGetShaderPrecisionFormat(GL_FRAGMENT_SHADER, GL_HIGH_FLOAT, (int *)&b, &a);
printf("Floating point bits of precision: %d\n"
       "Lowest value: -2e%d\n"
       "Highest value: 2e%d\n", a, b[0], b[1]);

Vendor: NVIDIA Corporation
Renderer: NVIDIA GeForce RTX 2070
Version: 4.6.0 NVIDIA 515.76
Shader language: 4.60 NVIDIA
Floating point bits of precision: 23
Lowest value: -2e127
Highest value: 2e127

The repository is riiiiiiight here. Star it on Github here too, I’ll need them.

GOODBYE.