Batch Update

I haven't been posting recently because I've been focusing on other areas of my life, but there has still been progress with PsyLink:

Prototype 6 improvements

I redesigned Power Module 3 to be more easily hand-solderable by enlarging the solder pads and adding thermal clearances to some of the filled copper areas. (The copper tracks to several pads were so wide that the heat of the solder iron was dissipating too fast into the copper, and the pads were not heating up properly). The result is Power Module 3.1.

Prototype 7

PerlinWarp reached out to me, and after some fruitful conversations, I build a Prototype 7 for him for research purposes. It's a hybrid of Prototype 4 and Prototype 6 which combines the advanced Power Module 3 with a cluster of Electrode Module 1 attached to Sleeve 3.

The assembly of the Prototype 7 took a full 20 hours of mindless labor, so I had a LOT of time to think about improvements to the design that would make it easier to assemble:

• Instead of having small boards for differential amplifiers that have to be tediously connected with three fragile wires, bulk four of them up in a single board.
• Instead of having a piece of custom-tailored fabric with precisely placed electrodes, use electrodes made of metal screw-like parts that fit right on to the PCB. The PCBs are held together with a rubber band that presses the electrodes tightly to the skin.
• Made Power Module 3 more easily hand-solderable (described above)

Prototype 8 (WIP)

The 8th prototype implements those changes, resulting in a device purely made of PCBs connected by rubber bands and wires. These PCBs rest on the skin on legs of metal, which double as the electrodes.

As of now, only the differential amplifiers are finished. The circuit for these is Circuit 10 (electrodes):

The front-side of the Electrode Module 3:

The four wide holes in the board are connection points for rubber bands, and the eight round metal-plated holes are mounting points the electrodes. These consist of a screw on one side, a nut on the other, holding both metal pieces tightly in place.

The electrodes can be spacer screws like these or dome nuts like these. While the spacer screws seem to pick up a better signal, the dome nuts are considerably more comfortable thanks to the rounded cap, so I think I'll stick with these for now.

Dome nuts are also called "hutmutter" in German, which translates to "hut's mother" (my nickname is hut), so just because of that fact, I had to try them out.

Additionally, one can use the pins EX1 through EX8 to connect arbitrary external electrodes.

A first prototype using Electrode Module 3 and Power Module 3.1 (with an improvised rubber band mounting mechanism) can be seen here:

I played around with using off-the-shelf wet EMS (electric muscle stimulation) electrodes for the ground electrode, since good connectivity for the ground electrode is important for a good signal.

I did not find an improvement in the signal though, perhaps wet electrodes are not worth it.

Finally, the global microchip shortage has affected this project as well, and I could not find any sources for the power converter chips (TPS61220 DCKR) that boost the battery voltage of 1.2V to the required 5.0V. I still have some legit chips left, but in the long term, I need more of these, so I acquired some chips from Shenzhen via AliExpress. They have been advertised as "Original new, 100% quality", so what could POSSIBLY go wrong? But I tried them out, and they actually do their job quite nicely.

Future plans

The power module is currently only clumsily attached to the rubber band. The finished Prototype 8 will need a power module that's more compatible with the Electrode Module 3, by making it as long as the Electrode Module 3, and either by adding rubber band mounting holes, or by adding pin headers so that it can be stacked on top of the amplifier board, which would reduce the number of dangling wires as well.

I will also try out cheaper differential amplifiers to reduce the total cost of the device from ~70€ to ~30€ (excluding the ~39€ for the Arduino Nano 33 BLE Sense).

Neurofeedback: Training in both directions

For now, the training of neural networks mostly happens on the AI side. The human makes arm gestures and presses keys on the keyboard. This provides the input (electrode and IMU signals) as well as the output/labels (keyboard actions) to the artificial neural network, which then learns the correlation between the two through supervised gradient descent.

Now I'm looking into how to train both the neural network of the AI as well as the nervous system of the user through neurofeedback, that is, by making the user more aware of their neural signals, which in turn allows them to fine-tune these.

My hope is that this will make up for the low quality of information that's available to the AI, due to noise, attenuation, and the low number/quality of electrodes. The user neither knows what signals the electrodes can access, nor how to willingly produce movements that create these signals. Some gestures work well, while others can't be detected at all, so the best bet is to use forceful gestures with maximal muscle activation. But if there was some sort of feedback to the user, like a visualization of the data that the neural network is extracting, the user could focus on the movements that work, and gradually lower the intensity, perhaps to the point where no actual movement is required anymore.

Of course there is already some feedback about the signals: The PsyLink UI shows the amplitude of each electrode in a rolling graph, and the GNURadio application shows detailed plots of the raw signals, both of which already help determining which movements will work for gestures and which will not. But the AI can of course combine, cross-correlate, filter, convolve and deconvolve the signals, which enables it to extract information that a human won't see in the raw signal data.

Ultimately, the goal is that the user learns to, on demand, fire off just enough neurons that PsyLink can pick up the signal and trigger the intended key press without any visible movement of the arm.

Approach

As described above, simply presenting the user with the raw electrode data is insufficient. A machine-learning approach will likely be optimal here, to overcome the preconceptions of a top-down designer. Since we already have an artificial neural network, why not use that one to generate the visualizations too?

In my current version of this idea:

1. The user needs to invent some arbitrary gesture that should correspond to the action "Press key 'A'".
2. The user is repeatedly asked to perform the gesture by the UI
   • At random intervals
   • For random durations
   • With 2-3 seconds of heads-up warning to account for reaction time
   • In between the gestures, the user should perform random other activities, but never do the gesture without being asked by the UI
3. The AI is trained on the fly with
   • Electrode signals as input
   • A binary label of "Key 'A' pressed" vs. "Key 'A' not pressed" as output
   • Each data point is added randomly (80:20) to the training or validation dataset
   • After X seconds of collecting data, the AI is trained for Y epochs
4. Every Z milliseconds, the AI is asked to predict the output from the current input, and the neural activations of the last non-output layer of the NN are presented to the user visually, along with the predicted output.
   • The visualization could be a heatmap or a scatterplot, for example
   • The visualization should cover a large dynamic range (both small changes and large changes to the values should be easily visible)
5. Using the feedback, the user can tweak their gesture as desired, to e.g.
   • Minimize the movement required to trigger the key
   • Maximize the reliability with which the key press is predicted
6. Over time, old data is dropped from the NN training to refine the visualization and to keep the training time short.

Once the user is ready, they can add a second action like "Press key 'B'" and so on.

New Frontpage + Logo

The front page now looks a little more "modern", and I changed the logo from

to

Once again, this was inspired by the System Shock 2 Trioptimum logo.

Cyber Wristband of Telepathy +2 [UNIQUE ITEM]

Power Module 3 finally found a home: Sleeve 4. Looks a bit like a Pip-Boy from the Fallout Series :)

Under the hood there's 9 electrodes (1x ground, 8x signal):

Running on AAA battery

Wow, it feels like ages since I started working on making PsyLink run on a rechargeable AAA battery. It sounds so simple and straight-forward, but it wasn't :). This ate 2.5 weeks of my time, but finally I succeeded!

Coming from software engineering, the iteration time of hardware prototypes is horribly slow. There was a lot of waiting for package deliveries, a lot of time assembling, and a lot of wrestling with leaky abstractions. For example, it wasn't enough to just connect the 1.2V->5V boost converter like on the circuit diagram, but I had to take special care of the distances between the parts, and the widths of the copper tracks connecting them.

Another problem you never face in software engineering is that the package with electronics parts was stolen, and when the new PCB of Power Module 3 arrived, I had to work with the few remaining (suboptimal) parts that I had.

What almost drove me insane was that I had only one fresh TPS61220 chip left. That's the 5V boost converter at position U1 (in the bottom of the red circle on the photo below), which is so small that I don't really have the tools to solder it on properly. I kept accidentally connecting the pins of the chip with solder. I gave up and started asking around friends for whether they could solder it on for me, when I remembered this soldering tip someone gave me: If you don't have soldering flux (which I didn't have), just use ✨margarine✨ instead ;D. It sounds very wrong, but it actually made a huge difference, and only thanks to the power of margarine I was able to keep the solder exactly where I needed it to be. "If it's stupid and it works, it's not stupid."

Then - for whatever reason - the voltage was going up to 1.5V instead of 5V, just like with the previous PCB layout. I thought I fixed that problem by optimizing the layout around the TPS61220 chip, but apparently that wasn't enough. I figured the 4.7µH inductance at L1 wasn't big enough, so I squeezed in a second coil:

(It looks like two tardigrades playing ball :) I wish I made a better picture before I disassembled it again)

This alone didn't help, but when I manually held an additional 10µH inductance in parallel to L1 for 1 second while the device was running, it surprisingly kickstarted and reached 5V!... until I drained some current, which made the voltage collapse back to 1.5V immediately. Curious behavior. :D

Then I tried soldering on the big 10µH coil directly onto the SMD pads, and the voltage reached 5V and stayed at 5V =)

No idea why the big 10µH coil worked while the two smaller coils totaling 9.4µH didn't work... I doubt that the 0.6µH difference in total inductance turned the tide, probably there's some factor I'm not aware of. I actually had ordered a 10µH SMD inductance coil in anticipation of this, but well, it got stolen... My only consolation is the face of the package thieves when they open the package, they realize that it's just a couple of tiny SMD parts, and they wonder WTF this shit is even good for.

P.S.: I found out that the reason why the coils didn't work was that they were not rated for the >200mA that's passing through them. Once I got a 4.7µH coil rated for 280mA, everything was fine.

More pictures:

The lack of parts forced me to adjust the resistance/inductance/capacitance values of Circuit 9 (power):

Thankfully I had planned for way too many capacitors on the PCB, just for some extra VROOOM, so the fact that I only had 10µF capacitors instead of the planned 100µF didn't matter too much.

I also changed the 1MΩ resistors in the reference voltage generator to 110kΩ because I read that smaller resistances in a voltage divider make the output voltage more stable, at the cost of more power use, but I think we can sacrifice some power for signal accuracy. No idea whether it's actually going to help though. I chose 110kΩ instead of the more common 100kΩ and 220kΩ because the boost converter already requires a 110kΩ resistor, and that way this prototype requires fewer different parts. But the exact values don't matter, as long as both resistances are identical.

Tomorrow I'll start working on the electrodes and wristband. :)