Purposes of this demo

Why would one want to render HTML using WebGL? Besides flashy effects and per-pixel manipulations possible by writing shaders, this method is useful where performance and responsiveness are UI critical.

If we are to build UI that required a stack of layered cards (as in Tinder), which can be fluidly moved by touch and faded out, layouting, compositing and capturing dragging events at the same time can be pretty costly.

Compositing large elements with a lot of children which may have have their own opacities / shadowes, all while capturing touch events can get real bad real soon

What if we could create a WebGL context, and render each card as a simple quad with a texture of it's corresponding UI, instead of trying to render complex tree structure such as the DOM? Our rendering will be offloaded to the GPU, resulting in a massive performance boost and per-pixel control over its graphics using shaders.

This technique works nicely with React and more specifically ReactDOMServer.renderToString(). It can be offloaded to web-workers for string concatenation and offscreen canvas for rendering.

It all starts with our input HTML

We will insert our specially formated HTML into SVG <foreignObject> special tag. It uses the built-in browser rendering pipeline to display HTML as part of a SVG graphic. We will then draw the SVG as a regular image onto a canvas2d element, which in turn we will use as a source to webgl texture.

Restrictions of this technique

There are some restrictions to what kind of HTML is allowed though:

1. No external assets, such as images, fonts, styles etc are allowed. If we want to display an image or use a special font, we need to download it first, encode it into base64 and then supply it to the<foreignObject>.
   This can quickly cause massively big strings to work with.
2. We have to inline all of our styles and be careful to prevent any styling inheritance on the element we want to render and its children.
   
   That's because even though our HTML and SVG can inherit the host page stylesheets, once we pass them to the canvas for drawing, these inherited styles will be stripped and not applied.
   (If you inspect this demo page itself, you will see how specific I am with my css selectors. I do not apply any global style to specific tags to prevent non-transfered styles once I rendert to canvas)
   
   We can either:
   
   
   1. Inline all of our styles directly on the HTML to be drawn (this is what I do in this demo, you can inspect step 1 contents): <div style=" position: relative; font-family: Arial, Helvetica, sans-serif; max-width: 400px; padding: 20px 30px 20px 20px; display: flex; justify-content: space-between; "> ... </div>
   2. Compute the styles that affect each child in our HTML using getComputedStyle() and append the result to an inline tag. This can be a huge performance bottleneck and will result in giant strings too (you have to apply every possible style that affect this element to make sure it renders correctly)
3. Since our final result is a bitmap graphic rendered in WebGL, we will obviously lose real selectable text, any SEO and any kind of mouse, keyboard, etc events. We would need to supply our own logic to decide on which element we are, in fact, hovering or clicking.

But it's not all that bad

Besides these deal-breakers, it seems browser vendors did not write a custom rendering path for the <foreignObject> and everything else appears to be fair game. Flexbox, various elements, fancy CSS transforms and even other SVGs can all be embeded and render just fine.

TODO:

• Safari has problems with inlined SVG whitespace and is not rendering the images