demo

A WebGPU demo of Growing Neural Cellular Automata (Mordvintsev et al., 2020). Each cell in a 40x40 grid runs a shared pretrained neural network in a compute shader to update its state. The CA grows from a single seed cell and self-repairs when damaged.

Cell State and Perception

Each cell has a 16-channel state vector: RGBA + 12 hidden channels. The simulation seeds a single center cell (RGB=0, alpha and hidden channels = 1) and lets the network grow outward from there.

Each cell builds a 48-dim input by convolving its 3×3 neighborhood with three fixed kernels: identity, Sobel-X, Sobel-Y across all 16 channels. The Sobel filters give the network gradient information without requiring it to learn spatial awareness from scratch.

Network and Update Rule

Two linear layers output a delta added to the current state. Network weights are uploaded to GPU storage buffers once and shared read-only across all cells per step.

Two mechanisms from the paper govern the simulation: stochastic updates (each cell fires with 50% probability) and a life mask (cells with no alive neighbors are zeroed out).

WebGPU Pipeline

Compute and render passes run back-to-back each frame. Double-buffered state avoids read-write collisions.

You can draw on the canvas to erase cells. The pattern self-repairs: a core property of the neural CA.

Confused? See Part 1.

Remember that time when I cracked the Google random number generator? Rumour has it that it stopped working: I’ve been told that the code is slow, the numbers are wrong, it ruined someone’s well-rehearsed party trick in front of their crush, bankrupted a small business selling “intricate magic boxes”, caused a train delay and consequently a minor, albeit heated, diplomatic incident, among other things.

Recap

The original method recovered the 128-bit xorshift state from consecutive Math.random() outputs by feeding a constraint system into Z3. Two things have since changed: how the internal state is converted to a double and the xorshift algorithm itself.

Change 1: ToDouble

Commit e0609ce6:

Make random ToDouble better distributed
Use division instead of bit magic plus subtraction to map the RNG integer state into the [0,1) double value range.

The old ToDouble produced floats in [0, 1) using “bit magic”:

static inline double ToDouble(uint64_t state0) {
    static const uint64_t kExponentBits = uint64_t{0x3FF0000000000000};
    uint64_t random = (state0 >> 12) | kExponentBits;
    return base::bit_cast<double>(random) - 1;
}

The new version replaces this with integer division. By the way, it’s a great case to see the difference between bit_cast and static_cast:

static inline double ToDouble(uint64_t random) {
    // Get a random [0,2**53) integer value (up to MAX_SAFE_INTEGER) by dropping
    // 11 bits of the state.
    double random_0_to_2_53 = static_cast<double>(random >> 11);
    // Map this to [0,1) by division with 2**53.
    constexpr double k2_53{static_cast<uint64_t>(1) << 53};
    return random_0_to_2_53 / k2_53;
}

In the Python code it’s mirrored as:

def to_double(random):
    return (random >> 11) / (1 << 53)

Right-shifting by 11 bits, instead of 12, gives us one extra bit of entropy, making it 53 bits total and a statistically better, more uniform distribution.

This changes how the bounds are set. Recall that the observed value x is produced by:

x = Math.floor(Math.random() * (max - min + 1)) + min

So the 53-bit integer r satisfies:

floor(r / 2^53 * (max - min + 1)) == x - min

r maps to x when it falls in the half-open interval [lo, hi), where lo = normalize(x) * 2^53 and hi = normalize(x+1) * 2^53. Since r is an integer, the inclusive bounds are [ceil(lo), ceil(hi) - 1]. Using floor(lo) would include a value that maps to x - 1, so ceil(lo) is used. Similarly, ceil(hi) - 1 is used because ceil(hi) would map to x + 1. In the code:

def normalize(val):
    return (val - min) / (max - min + 1)

def get_bounds(point):
    lo = normalize(point) * (1 << 53)
    hi = normalize(point + 1) * (1 << 53)
    return math.ceil(lo), math.ceil(hi) - 1

The nasty exponent edge-case check from the original, where upper_exp == 1024, disappears entirely.

Change 2: xorshift128+ with a plus

Commit 0596ead5:

Resurrect add operation in xorshift128+ implementation
… which was accidentally removed in https://crrev.com/c/1238551.

For 7 years, the Korova milk bar served plain milk instead of milk plus

This one is remarkable. Since 2018 xorshift128+ wasn’t actually what it claims. It was plain xorshift128 without the plus. The 64-bit states weren’t summed and only state0 was used to fill the cache:

// Old: void, return value discarded
static inline void XorShift128(uint64_t* state0, uint64_t* state1) {
    uint64_t s1 = *state0;
    uint64_t s0 = *state1;
    *state0 = s0;
    s1 ^= s1 << 23;
    s1 ^= s1 >> 17;
    s1 ^= s0;
    s1 ^= s0 >> 26;
    *state1 = s1;
}

XorShift128(&state.s0, &state.s1);
cache->set(i, ToDouble(state.s0));

After 7 years in production, this was fixed by returning the sum s0 + s1:

// New: returns s0 + s1
static inline uint64_t XorShift128(uint64_t* state0, uint64_t* state1) {
    uint64_t s1 = *state0;
    uint64_t s0 = *state1;
    *state0 = s0;
    s1 ^= s1 << 23;
    s1 ^= s1 >> 17;
    s1 ^= s0;
    s1 ^= s0 >> 26;
    *state1 = s1;
    return s0 + s1;  // <-- this line was missing for 7 years
}

uint64_t random = XorShift128(&state.s0, &state.s1);
cache->set(i, ToDouble(random));

Now the output is state0 + state1 so the constraints become:

points = points[::-1]
for point in points:
    state0, state1 = xs128p(state0, state1)
    output = state0 + state1
    calc = z3.LShR(output, 11)

    lower, upper = get_bounds(point)
    solver.add(z3.And(lower <= calc, calc <= upper))

Both state variables are entangled in every constraint, which increases computation time.

In the prediction itself we also use the sum of the states:

if solver.check() == z3.sat:
    model = solver.model()
    state = {}
    for d in model.decls():
        state[d.name()] = model[d]
    s0_val = state["state0"].as_long()
    s1_val = state["state1"].as_long()

    output = (s0_val + s1_val) & 0xFFFFFFFFFFFFFFFF
    next = math.floor(to_double(output) * (max - min + 1)) + min
    print(next)

Here, overflow will happen during summation, so a 64-bit mask (0xFFFFFFFFFFFFFFFF) is applied to the output.

The observer’s paradox

Someone's sipping my entropy

For ease of collecting data, I logged generated points into the console and copied into my code. It worked. However, later, when I tested it again, but copied the numbers manually, the solver failed. I tried again with the data from the console: it worked. Although the numbers were the same, when I copied them manually from the widget, it failed. Weird.

Quantum computing? No, telemetry. The paradox was caused by a call to Math.random() triggered by an event listener on selection, attached to the whole document. It’s connected to Google’s telemetry and Math.random() is used to collect just 5% of the data.

Moreover, I’ve discovered that when using Chrome without a Google profile logged in, every click of the “Generate” button runs 2 extra calls of Math.random(). Similarly, some other clicks also trigger 2 calls. This is connected to telemetry too, which requires a 64-bit ID, generated by two halves consuming one call each.

Although it’s possible to handle this case in the solver by adding extra runs of xorshift for every point without any constraints, I’ve decided not to do it, due to the unclear nature of these twisted calls and my general disdain for them. They are more unpredictable than the RNG itself.

As a result, in my case, the solver works as-is in Google Chrome v147 with a Google profile without selecting anything on the page in between calls. Or more generally: for any sequence of consecutive Math.random() calls.

Perhaps the electron, too, behaves differently under observation because of all-encompassing telemetry calls.

Code: https://github.com/IvanLudvig/google-rng-hack

I arrived in Orsomarso for a day trip. It was just me and an older lady on the morning bus. The village had a wintery feel: it was quiet, chilly and smelled of chimney smoke. It’s almost 9AM. What a convenient place to put a clock, by the way.

Clock tower above the village

Leaving the village for a hike, I encountered welcoming animals. The puppies were playful.

Puppies

The fluffy cat looked wise and polite.

Wise cat

The river is called Argentino, meaning “silvery”. I would say that the stream is clear blue.

Clear blue stream

The waterfall breaks the silence. Otherwise the path is absolutely quiet, peaceful and solitary.

Waterfall

Shrine

On the way back to Orsomarso, I encountered a prominent rock. I’m sure there are legends about it, probably not good ones.

Prominent rock

Approaching the village

Orsomarso has a long history with Greek heritage: the valley once had Eastern Orthodox monasteries and hermits’ caves (see Mercurion). In one of the nearby caves, above the village, Saint Nilus lived as a hermit.

View from clock tower

Now, a lonely statue of the Virgin Mary stands in another big cave.

Lourdes Grotto

Guimarães is called the birthplace of Portugal. I visited it to build a more personal connection to the country. I imagine right here, in this gothic canopy, the country was born. I think it cried a lot as a child.

Oliveira square

The old town is beautifully preserved and has a medieval atmosphere. Turns out, Portugal was always gloomy.

Old town

Santos Passos Church

Tiled picture

The castle somehow looks out of place: a brutal building on a green lawn. Then again, it is supposed to be intimidating. But there’s just something too perfect about it, that makes it seem unreal. I wish I had seen it at night too.

Guimarães Castle

It seems appropriate that Portugal grew up to build solemn monuments.

Solemn monuments

Coimbra

App

Ethereum Blockie Search is a tool to generate Ethereum addresses with specific patterns in their Blockie 8x8 identicons.

Developed during a small vibe-coding hackathon in Porto.

Coimbra is monumental and classy.

City above the river

I wonder what the river is called. Oh, maybe Torga? I actually believed this, before looking it up: it’s Mondego. Torga is a writer. Either way, the grass makes the sign blend in nicely.

Torga sign

The statue also has a label: King João III. Seems like it needs cleaning though.

Statue of King João III

The railway infrastructure in Portugal has a nice minimal design with labels against a plain white background.

Train

Looking at the post again, I realised that it doesn’t capture Coimbra, but only a variety of labelled objects. Sometimes even my blog fails to deliver, so as a consolation here’s a fabulous key shop sign.

Key shop

Figueira da Foz is a dreamy slow place. A fading resort town out of season.

Dreamy town

Seafront

The beach is big in both dimensions so it takes a long time to walk to the ocean.

Ocean

Figueira has an interesting example of the Portuguese pavement with a jagged pattern, which I haven’t seen before. It somehow fits this place.

Portuguese pavement

This pale white sunset with a hint of pink reminded me that it’s actually winter. Soft cold colours, gloomy light, trees without leaves. It feels like life has paused. It’s not sad. There are birds. Birds are always happy.

Winter sunset

This post captures what Figueira actually is. Still, I want to include a shopfront, even two.

Shopfronts

As we all know, everything ends in darkness. Might as well light up the Christmas tree. It’s December.

Night

Porto

Guimarães

When people asked me why I came to Porto, I told them that I didn’t want to come specifically to Porto, but my flight was here and I didn’t mind staying.

I’ve been here before, last year. This time, I spent a couple of weeks in the city.

Have you visited Porto? Why?

Sao Miguel

Coimbra

In my hostel in Ponta Delgada there was a hand-written guidebook of the island. I encountered it on the same day, but 7 years later and sat on the same spot as illustrated. I love things like this.

Hand-written guidebook

On a sunny day, Ponta Delgada has a lot of contrast: the buildings and pavement tiles are black and white, made from dark black volcanic rock.

City gates

Igreja do Colégio dos Jesuítas

Peaceful morning

Cute church

I was impressed by the Antonio Borges Botanical Garden with birds and tropical-looking palm trees, which was free to visit. It’s like a piece of jungle in the city.

Botanical garden

I found it unexpected how the city was run down outside of the immediate centre. The narrow streets are uncomfortable for both people and cars. There are a lot of dilapidated buildings.

Streets

Like with the Canary islands in Spain, I was expecting the cities in the Azores to be nicer than mainland Portugal, but found it to be the opposite case. They seem like a sad place to be.

Favela vibe

Being without a car, I decided to join a tour to Sete Cidades and Mosteiros, offered by my hostel for 30 euros. Funny thing: the tour required hiking shoes but a girl did it in flip flops. I had no hiking shoes either.

Even though it’s sunny on the coast, it’s often foggy inland of the island. The path with one of the most iconic views was surrounded by thick fog. There’s supposed to be a lake down there, or even two.

Path

Luckily, the lakes were visible from a different viewpoint and there are in fact two, of different colours. You can also see the ocean behind the hill. Incredible.

Lakes

The misty roads around the lakes are like a fairytale.

Misty roads

There is a big abandoned hotel nearby. Empty in a remote place on a remote island.

Abandoned hotel

The town on the lake is quiet and grey, existing in a dreamy state. Nothing bad could ever happen here, surely.

Church

Stained house

Lake

Maybe the saying that goes “the grass is always greener…” actually refers to the isolated Azores. The grass is the greenest thing I’ve seen. Imagine how lucky are the cows. Imagine the milk.

Cows grazing on greenest grass

The parts of land that are formed by lava flows onto the coast have a name here: Fajã.

Cliffs

After a busy day we swam at the beach and had beer. Actually, it was only me swimming and nobody else. The beach is on the West of the island and I imagined how small and isolated are the islands, and how if I were to get carried away by the waves, the next big piece of land was going to be North America far, far away.

Mosteiros beach

I love the dark churches made from volcanic rock. They look ominous. The cross has an “S.P.Q.R.” inscription. I wonder what meaning was put into it given the location.

Igreja Matriz de São Miguel Arcanjo

Pharmacy

Up on the hill, above the town, is the sanctuary of Nossa Senhora da Paz.

On the way

The church’s cascade reminds me of the Bom Jesus Sanctuary in Braga, but smaller and lonelier. Actually, the whole island is lonely.

Sanctuary of Nossa Senhora da Paz

The legend

The views from the top are fantastic. I love the black volcanic cross.

View from the top

On another day, I made a small hike nearby around a waterfall, named Cascata do Segredo, which translates as “the secret waterfall”. I found the name to be very fitting. In fact, having discovered just about any waterfall, I would’ve been inclined to call it that way. Waterfalls are secret, intimate places.

I enjoyed the lush green forest, encountered unusual plants and swam in the cool water. It felt very fresh and enriched my understanding of the island’s nature, which I still didn’t fully grasp. It’s both tropical and northern.

Cascata do Segredo hike

The trail passed through remains of an abandoned hydroelectric station, but I didn’t care for it at all. I never understood what draws people to abandoned structures unless they’re very old.

Abandoned hydroelectric station

Village

Furnas is full of volcanic hot springs, which makes the whole town smell particular. In the Caldeiras park you could also see and hear them. There was a nativity scene arranged around the springs, right in the middle of the steam.

Caldeiras park

At the water fountains you can taste them. The water has a pronounced metallic taste and slightly carbonated, feeling electric. Interestingly, the fountains are labeled with exact temperature. Some of them are very hot.

Water fountain

Finally, the ultimate experience is bathing in the springs. I visited the smaller Poça da Dona Beija thermal park. It was warm and cozy especially with the colder weather. I also loved greenery around and recall the distinct colour combination of red iron and all sorts of green plants.

Poça da Dona Beija thermal park

Furnas has hints of green in the town structures too.

Green tint

Finally, I walked to the lake. It was the first time I walked through a bamboo forest. I also loved the solemn grey gothic church.

Furnas lake

After visiting the lake, I hitchhiked back to Ponta Delgada. I waited 40min for a ride and joined the driver on a side quest to get a ridiculously large amount of water from one of the fountains. I filled my modest water bottle too. This took quite some time, so I admired the sunset out of the car window. This was my last night on Sao Miguel.

Hitchhiking

After a week, I left the isolated green island. Lush and alive but alone in the middle of the ocean.

Gijón

Porto

Gijón is a coastal city just 30km from the capital of Asturias, Oviedo. They are of comparable sizes and I chose to stay in Gijón because I prefer the sea. The city sits against green hills and snowy mountains further away, those are part of Picos de Europa.

City against the mountains

My BlaBlaCar driver wondered why I’d come here and said the city is industrial and not nice. At least I was reassured about my Spanish skills because our long conversation was in Spanish.

Indeed, I found Gijón quite unremarkable at first glance. The city feels provincial.

Streets

The bus station has a slick old sign. Unfortunately, it lacks any other relevant signs.

Bus station

A piece of street art with overlapping faces on a weathered wall touched me. I associate it with something deeply human: the quiet desperation of realizing the fragility of life.

Faces on a wall

Windows facing the ocean form an ocean of their own, that flows from building to building.

Windows

The ocean is fantastic and the city has a nice long beach. Even surfers come here, or at least one did when I was there.

Ocean

It’s possible to walk far along the coast. Different monuments and sculptures, often modern, are placed along it.

Coast

Such as Elogio del Horizonte. When I first encountered a similar coastal sculpture in Donostia, Comb of the Wind, I didn’t understand it. The concrete and rust, evoking a feeling of burden – why is it there? Gijón was the last city on my Northern Spain trip and I finally got it.

Elogio del Horizonte

My head spun from it.

Experiencing art

Santander

Sao Miguel