PsyLink
2021-12-15

Power Module 4

Prototype 8 needs a matching power module. I can't continue to awkwardly stuff the rubber band between the velcro and the PCB of Power Module 3 like this:

showing a p8 with a very awkward rubber band construction

We need a PCB with proper rubber band holes and screw mounting points, so it can be worn along the radius bone with about 2x Electrode Module 3 beside it.

Unfortunately the large dimensions of the Arduino and the battery make it a little difficult. There's several options to do this, but none of them are perfect:

  1. Place Arduino and battery beside each other like in Power Module 3 and either...
    1. make the board ~15mm longer to fit the holes.
    2. make the board ~12mm wider to fit the holes.
      • Downside: It would be too wide to comfortably fit on the radius bone
    • Downside of both options: it might feel unbalanced when worn on the side of the arm, because the heavy battery moves the center of mass off to the side
  2. Place Arduino and battery on top of each other, with pin sockets for the arduino that are taller than the battery, then make the board the size of Electrode Module 3 to fit the holes
    • Downside: It might be hard to acquire pin sockets that tall
    • Downside: Little space left for the holes and the remaining components, but might work
  3. Place Arduino and battery in a line next to each other and make the board 20mm longer and 8mm wider than Electrode Module 3 to fit the holes.
    • Downside: The rubber bands are relatively close to the center, reducing the force with which the far ends of the board are pressed onto the skin
    • Downside: The large dimensions might make it too bulky
  4. Remove the Arduino and solder the NINA-B306 bluetooth module of the Arduino onto the board directly, along with all the peripherals I need
    • Downside: Unnecessary complexity
    • Downside: More difficult soldering
  5. Instead of adding any holes, just stack the board on top of a Electrode Module 3 and use its screws and rubber band holes for physical support. (Thanks to PerlinWarp for that idea!)
    • Downside: Would require a redesign of Electrode Module 3
    • Downside: The height of the prototype might make it feel unbalanced
    • Downside: The board would have be either wider or longer than the Electrode Module 3 due to the dimensions of the Arduino and battery. As a result, it would stick out to the sides, unless I enlarge the electrode board as well.

In the end I chose option 3 for simplicity reasons, and because the downsides seemed the most acceptable. This also gives me a lot of space on the PCB which is currently not needed, but may be used for analog multiplexing later on, to allow supporting more than 8 electrodes.

I am quite tempted by option 2 though, it would make for a really compact board. Maybe I give it a try later (if I ever find a source for 1x15 pin sockets that are 14mm+ tall and 2.54mm spaced apart.)

Here is the resulting board. As you see, I also added a screw mounting hole in the center, which should point right at the bone, which perhaps makes for a great ground electrode. Any of the screws can be used as ground electrodes though, simply by closing their respective solder jumper. I will try out various combinations to see which one is best.

PCB bp4 front side

PCB bp4 back side

The matching circuit:

circuit 11

I ordered the production of this board already, and I'm excited to see if it will work out like I imagine. :)

Fun story on the side: I ordered the boards on Aisler, and they let you change the PCB design for free until it hits production. This time, I had a couple of hours left, which I used to make a couple of adjustments here and there, until I was happy and went to sleep. After 3 hours I woke up, with racing thoughts, and I couldn't really fall asleep again (which happens very rarely). After laying awake for 2 hours, it occurred to me that I should run the automatic PCB layout check of KiCad once again to test my board for obvious mistakes. Of course I had run the check before submitting my order, and several times after doing my various modifications. But as it turned out, I had indeed forgotten to run it on the final version, and there were multiple errors too! Several unconnected nodes and some copper tracks that were too close to each other. Luckily I caught this just a couple minutes before my time for free modification ran out, and fixed it just in time. Thanks insomnia.

2021-11-30

Batch Update

I haven't been posting recently because I've been focusing on other areas of my life, but there has still been progress with PsyLink:

Prototype 6 improvements

I redesigned Power Module 3 to be more easily hand-solderable by enlarging the solder pads and adding thermal clearances to some of the filled copper areas. (The copper tracks to several pads were so wide that the heat of the solder iron was dissipating too fast into the copper, and the pads were not heating up properly). The result is Power Module 3.1.

Prototype 7

PerlinWarp reached out to me, and after some fruitful conversations, I build a Prototype 7 for him for research purposes. It's a hybrid of Prototype 4 and Prototype 6 which combines the advanced Power Module 3 with a cluster of Electrode Module 1 attached to Sleeve 3.

p7

The assembly of the Prototype 7 took a full 20 hours of mindless labor, so I had a LOT of time to think about improvements to the design that would make it easier to assemble:

Prototype 8 (WIP)

The 8th prototype implements those changes, resulting in a device purely made of PCBs connected by rubber bands and wires. These PCBs rest on the skin on legs of metal, which double as the electrodes.

As of now, only the differential amplifiers are finished. The circuit for these is Circuit 10 (electrodes):

c10

The front-side of the Electrode Module 3:

b7

The four wide holes in the board are connection points for rubber bands, and the eight round metal-plated holes are mounting points the electrodes. These consist of a screw on one side, a nut on the other, holding both metal pieces tightly in place.

The electrodes can be spacer screws like these or dome nuts like these. While the spacer screws seem to pick up a better signal, the dome nuts are considerably more comfortable thanks to the rounded cap, so I think I'll stick with these for now.

Dome nuts are also called "hutmutter" in German, which translates to "hut's mother" (my nickname is hut), so just because of that fact, I had to try them out.

Additionally, one can use the pins EX1 through EX8 to connect arbitrary external electrodes.

A first prototype using Electrode Module 3 and Power Module 3.1 (with an improvised rubber band mounting mechanism) can be seen here:

outside view

inside view

I played around with using off-the-shelf wet EMS (electric muscle stimulation) electrodes for the ground electrode, since good connectivity for the ground electrode is important for a good signal.

outside view with wet electrode

I did not find an improvement in the signal though, perhaps wet electrodes are not worth it.

Finally, the global microchip shortage has affected this project as well, and I could not find any sources for the power converter chips (TPS61220 DCKR) that boost the battery voltage of 1.2V to the required 5.0V. I still have some legit chips left, but in the long term, I need more of these, so I acquired some chips from Shenzhen via AliExpress. They have been advertised as "Original new, 100% quality", so what could POSSIBLY go wrong? But I tried them out, and they actually do their job quite nicely.

Future plans

The power module is currently only clumsily attached to the rubber band. The finished Prototype 8 will need a power module that's more compatible with the Electrode Module 3, by making it as long as the Electrode Module 3, and either by adding rubber band mounting holes, or by adding pin headers so that it can be stacked on top of the amplifier board, which would reduce the number of dangling wires as well.

I will also try out cheaper differential amplifiers to reduce the total cost of the device from ~70€ to ~30€ (excluding the ~39€ for the Arduino Nano 33 BLE Sense).

2021-07-17

Neurofeedback: Training in both directions

For now, the training of neural networks mostly happens on the AI side. The human makes arm gestures and presses keys on the keyboard. This provides the input (electrode and IMU signals) as well as the output/labels (keyboard actions) to the artificial neural network, which then learns the correlation between the two through supervised gradient descent.

Now I'm looking into how to train both the neural network of the AI as well as the nervous system of the user through neurofeedback, that is, by making the user more aware of their neural signals, which in turn allows them to fine-tune these.

My hope is that this will make up for the low quality of information that's available to the AI, due to noise, attenuation, and the low number/quality of electrodes. The user neither knows what signals the electrodes can access, nor how to willingly produce movements that create these signals. Some gestures work well, while others can't be detected at all, so the best bet is to use forceful gestures with maximal muscle activation. But if there was some sort of feedback to the user, like a visualization of the data that the neural network is extracting, the user could focus on the movements that work, and gradually lower the intensity, perhaps to the point where no actual movement is required anymore.

Of course there is already some feedback about the signals: The PsyLink UI shows the amplitude of each electrode in a rolling graph, and the GNURadio application shows detailed plots of the raw signals, both of which already help determining which movements will work for gestures and which will not. But the AI can of course combine, cross-correlate, filter, convolve and deconvolve the signals, which enables it to extract information that a human won't see in the raw signal data.

Ultimately, the goal is that the user learns to, on demand, fire off just enough neurons that PsyLink can pick up the signal and trigger the intended key press without any visible movement of the arm.

Approach

As described above, simply presenting the user with the raw electrode data is insufficient. A machine-learning approach will likely be optimal here, to overcome the preconceptions of a top-down designer. Since we already have an artificial neural network, why not use that one to generate the visualizations too?

In my current version of this idea:

  1. The user needs to invent some arbitrary gesture that should correspond to the action "Press key 'A'".
  2. The user is repeatedly asked to perform the gesture by the UI
    • At random intervals
    • For random durations
    • With 2-3 seconds of heads-up warning to account for reaction time
    • In between the gestures, the user should perform random other activities, but never do the gesture without being asked by the UI
  3. The AI is trained on the fly with
    • Electrode signals as input
    • A binary label of "Key 'A' pressed" vs. "Key 'A' not pressed" as output
    • Each data point is added randomly (80:20) to the training or validation dataset
    • After X seconds of collecting data, the AI is trained for Y epochs
  4. Every Z milliseconds, the AI is asked to predict the output from the current input, and the neural activations of the last non-output layer of the NN are presented to the user visually, along with the predicted output.
    • The visualization could be a heatmap or a scatterplot, for example
    • The visualization should cover a large dynamic range (both small changes and large changes to the values should be easily visible)
  5. Using the feedback, the user can tweak their gesture as desired, to e.g.
    • Minimize the movement required to trigger the key
    • Maximize the reliability with which the key press is predicted
  6. Over time, old data is dropped from the NN training to refine the visualization and to keep the training time short.

Once the user is ready, they can add a second action like "Press key 'B'" and so on.

2021-07-06

New Frontpage + Logo

The front page now looks a little more "modern", and I changed the logo from

old logo

to

old logo

Once again, this was inspired by the System Shock 2 Trioptimum logo.

2021-06-24

Cyber Wristband of Telepathy +2 [UNIQUE ITEM]

Power Module 3 finally found a home: Sleeve 4. Looks a bit like a Pip-Boy from the Fallout Series :)

photo, closed

Under the hood there's 9 electrodes (1x ground, 8x signal):

photo, open

2021-06-21

Running on AAA battery

Wow, it feels like ages since I started working on making PsyLink run on a rechargeable AAA battery. It sounds so simple and straight-forward, but it wasn't :). This ate 2.5 weeks of my time, but finally I succeeded!

photo, tilted perspective

Coming from software engineering, the iteration time of hardware prototypes is horribly slow. There was a lot of waiting for package deliveries, a lot of time assembling, and a lot of wrestling with leaky abstractions. For example, it wasn't enough to just connect the 1.2V->5V boost converter like on the circuit diagram, but I had to take special care of the distances between the parts, and the widths of the copper tracks connecting them.

Another problem you never face in software engineering is that the package with electronics parts was stolen, and when the new PCB of Power Module 3 arrived, I had to work with the few remaining (suboptimal) parts that I had.

What almost drove me insane was that I had only one fresh TPS61220 chip left. That's the 5V boost converter at position U1 (in the bottom of the red circle on the photo below), which is so small that I don't really have the tools to solder it on properly. I kept accidentally connecting the pins of the chip with solder. I gave up and started asking around friends for whether they could solder it on for me, when I remembered this soldering tip someone gave me: If you don't have soldering flux (which I didn't have), just use ✨margarine✨ instead ;D. It sounds very wrong, but it actually made a huge difference, and only thanks to the power of margarine I was able to keep the solder exactly where I needed it to be. "If it's stupid and it works, it's not stupid."

Then - for whatever reason - the voltage was going up to 1.5V instead of 5V, just like with the previous PCB layout. I thought I fixed that problem by optimizing the layout around the TPS61220 chip, but apparently that wasn't enough. I figured the 4.7µH inductance at L1 wasn't big enough, so I squeezed in a second coil:

photo, coil acrobatics

(It looks like two tardigrades playing ball :) I wish I made a better picture before I disassembled it again)

This alone didn't help, but when I manually held an additional 10µH inductance in parallel to L1 for 1 second while the device was running, it surprisingly kickstarted and reached 5V!... until I drained some current, which made the voltage collapse back to 1.5V immediately. Curious behavior. :D

Then I tried soldering on the big 10µH coil directly onto the SMD pads, and the voltage reached 5V and stayed at 5V =)

photo, front

No idea why the big 10µH coil worked while the two smaller coils totaling 9.4µH didn't work... I doubt that the 0.6µH difference in total inductance turned the tide, probably there's some factor I'm not aware of. I actually had ordered a 10µH SMD inductance coil in anticipation of this, but well, it got stolen... My only consolation is the face of the package thieves when they open the package, they realize that it's just a couple of tiny SMD parts, and they wonder WTF this shit is even good for.

P.S.: I found out that the reason why the coils didn't work was that they were not rated for the >200mA that's passing through them. Once I got a 4.7µH coil rated for 280mA, everything was fine.

More pictures:

photo, side photo, back

The lack of parts forced me to adjust the resistance/inductance/capacitance values of Circuit 9 (power):

circuit 9.1

Thankfully I had planned for way too many capacitors on the PCB, just for some extra VROOOM, so the fact that I only had 10µF capacitors instead of the planned 100µF didn't matter too much.

I also changed the 1MΩ resistors in the reference voltage generator to 110kΩ because I read that smaller resistances in a voltage divider make the output voltage more stable, at the cost of more power use, but I think we can sacrifice some power for signal accuracy. No idea whether it's actually going to help though. I chose 110kΩ instead of the more common 100kΩ and 220kΩ because the boost converter already requires a 110kΩ resistor, and that way this prototype requires fewer different parts. But the exact values don't matter, as long as both resistances are identical.

Tomorrow I'll start working on the electrodes and wristband. :)

2021-06-16

Power Module 3

Since Power Module 2 has the wrong PCB layout for the step-up converter, I built Power Module 3 to fix this issue.

It was still a worthwhile learning experience to build Power Module 2, resulting in the following changes:

Here's the circuit:

circuit 9

And the new PCB:

PCB bp3 front side

PCB bp3 back side

If you're wondering why the sparky, fancy looking power line goes all the way from the power switch on the right through digital pin D10 and into L1 on the left side... Indeed that looks pretty awkward, but L1 is the noisiest component, and I wanted to keep it as far as possible from the analog pins at the bottom, without sacrificing the effectiveness of the boost converter layout. Given the size & time constraints, I didn't find a better solution.

What time constraints, you ask? Well, actually I made a different PCB layout first. Polished every detail, and when it was perfect (according to my crude appraisal), I ordered it. Was already excited about the delivery, started putting the board on the website, and so on. But at some point I noticed that something was wrong... The Arduino pins were inverted. Theoretically, everything would still work, but the Arduino would have to be plugged in from the back side, which is something I wanted to avoid to keep the board laying nice and flat on the forearm...

Thankfully the manufacturing process hasn't started yet, and I could update the board for free. So I started redesigning half of the board and finished just in time for the production to start :D

Let's hope it works this time.

2021-06-10

Believe The Datasheet

Today the order of Power Module 2 arrived!

And with relief I saw that the battery clips fit nicely onto the board, as does the Arduino (with pin strips), and all the other components. Just that simple thing already felt like an accomplishment at my level of expertise with PCB design ;)

After some dreadful time trying to solder on the tiny 1x2mm-sized chip at U1 (I need a microscope for this shit), I had the SMD parts assembled and the circuit was ready for a test drive:

photo of assembled p5

But something was weird. The output voltage was a meager 1.5V, not the expected 5V, even though everything was connected properly. After hours of debugging I flipped over the table and just soldered a fresh board, this time without the Vref-generating OpAmp (U2+R3+R4+C3). But no luck, still just 1.5V. This was lower than the minimal output voltage of the voltage booster, so the chip didn't even finish it's start-up phase. How could that be, if there's not even any load on the output voltage?

I desperately tried several different things. One was doubling the inductance at L1 from 4.7µH to 9.4µH by using two SMD inductances in series:

photo

Unfortunately I think I broke the coils while constructing this, since they didn't let any current go through. But I found a regular, big inductance coil with 10µH, manually held its pins down onto the SMD pads, and indeed, the voltage jumped up to 5V!

So I soldered it... onto... the SMD pads.

photo

(Probably you can reconstruct my entire room from all the reflections in this image, along with a biometric picture of my face and at least 3 of my fingerprints...)

But once I connected the OpAmp, the voltage went back down to 2.5V, and with the Arduino connected, it went down further to 2.4V. Adding a second coil in series for a total of 20µH didn't compensate for this, but made it even worse, bringing it all the way down to 1.5V.

Well, clearly whatever is wrong with this construction has something to do with the inductance, and it's not purely the amount of inductance... Which brings us to the title of this post:

The mistake (probably)

Of course the data sheet of the voltage booster CLEARLY STATED that the inductance coil needs to be AS CLOSE AS PHYSICALLY POSSIBLE to the chip. The capacitors C1 and C2 too, by the way. And I even read that. But I thought, what could possibly go wrong if I move it ~1cm away to make some space for the battery? Nothing, right? Well, awesome, I guess it's time for another revision :)

At least the 2.4V were enough to power the Arduino, although it was visibly struggling. I could establish a Bluetooth connection and collect some signals, but the Bluetooth packets were coming in extremely slowly (though still faster than mobile internet in 80% of Germany.)

Pictures

Front:

Front

Somehow the oversized inductance coil adds a nice vibe to it.

Front, with battery:

Side

Back, with attached Arduino:

Back

This picture shows a pin strip socket that will be gone in the final version, where the Arduino will be soldered onto the board, reducing the height from 3cm to 2cm.

Side, with Arduino and battery:

Side

(Yes, the pin strip socket is too long by one pin ;))

2021-06-04

Back to the Roots

While uploading the old videos to the new PeerTube channel, I viewed the first video once again, which shows a pretty good signal from just two pieces of aluminum foil taped to the skin. And I wondered, why do I even bother with such a complicated set-up like in Prototype 4? It was really annoying to assemble, and the device is clunky and fragile.

Let's go back to the roots and build something more simple. Plenty of reasons:

Circuit 7

The Circuit 7 (whole device) shows a simplification of the signal processing module to a simple non-inverting amplifier per electrode with a gain of 221x. There's also a 560KΩ bias resistor towards Vref so the voltage we measure isn't too far off the center. In Circuit 6 (whole device) I had used 1MΩ, but here it produced mysterious oscillations, and going down to 560KΩ mysteriously fixed it.

This circuit also features a rechargeable 1.2V AAA battery with a TPS61220 step-up converter boosting the voltage to 5V, because I don't hate nature, and I burned through enough CR2032 coin cells. Coin cells also aren't exactly optimized for currents of 20mA, and thus get drained too quickly.

circuit 7

I measured a signal while pressing down a finger onto the table with two electrodes along the Flexor Digitorum Superficialis. Blue is electrode 1, red is electrode 2, and green is an amplified difference:

signals

Circuit 8

But let's cut even more out of this circuit. Here's one that is designed to be a shield to the Arduino Nano 33 BLE Sense, containing just the power supply, and an array of pass-through pins:

circuit 8

The signal that I'm getting is weaker, but certainly usable: (again, I measured a signal while pressing down a finger onto the table with two electrodes along the Flexor Digitorum Superficialis. Blue is electrode 1, red is electrode 2, and green is an amplified difference)

signals

PCB

I also built a PCB that implements this power supply/pass-through shield, and I figured, even if the device ends up not very useful, I'll still be able to use this for experiments later on, thanks to the pass-through pins.

bp2

I just hope that the PCB/circuit will work at all. I still haven't figured out how to simulate it, and I don't really know the best practices for PCB design. The PCB footprint for the AAA battery clips (Keystone 82) is my first custom-made PCB footprint too. Hope it all works out.

Next Prototype

It will be a relatively small forearm band with 8 electrodes (+ 1 ground electrode), which I plan to place around the Flexor Digitorum Superficialis for detecting what individual fingers are doing. The information from the gyroscope + accelerometer should cover the rest.

2021-05-31

Website is Ready

The website is more fleshed out now, with a nice black/green design, neurons in the background, and a logo that is based on the logo of the fictional TriOptimum Corporation from the System Shock game series.

Videos are now hosted on a PeerTube Channel, allowing me to upload higher quality videos in the future while keeping the git repository of the website small.

I also catalogued individual components (circuit schematics, circuit boards, textiles, software) that resulted from this project so far, and documented how they all fit together in the prototype overview. Each component has an individual ID now, that I can easily write, print or sew on the hardware so I don't mix everything up. For example, prototype 4 has the ID "p4" and can be reached directly via https://psylink.me/p4, while the signal processing board of p4 has the ID "b1" and can be reached via https://psylink.me/b1.

Here's a screenshot, for a future time when the design has changed:

screenshot of the new website

Page: 1 2 3 4 all