PsyLink is experimental hardware for reading muscle signals and using them to e.g. control the computer, recognize gestures, play video games, or simulate a keyboard. PsyLink is open source, sEMG-based, neural-network-powered, and can be obtained here.
- All posts on one page
2023-02-05: 2022 Retrospective
2022-02-24: Added Bills of Materials
2022-02-23: 3M Red Dot electrodes
2022-02-22: Microchip 6N11-100
2022-02-16: Next Steps & Resources
2022-02-15: Mass production
2022-01-19: Prototype 9 + Matrix Chatroom
2022-01-18: HackChat & Hackaday Article
2021-12-19: Prototype 8 Demo Video
2021-12-18: Prototype 8
2021-12-16: INA155 Instrumentation …
2021-12-15: Power Module 4
2021-11-30: Batch Update
2021-07-17: Neurofeedback: Training in …
2021-07-06: New Frontpage + Logo
2021-06-24: Cyber Wristband of Telepathy …
2021-06-21: Running on AAA battery
2021-06-16: Power Module 3
2021-06-10: Believe The Datasheet
2021-06-04: Back to the Roots
2021-05-31: Website is Ready
2021-05-29: Dedicated Website
2021-05-17: Gyroscope + Accelerometer
2021-05-14: Wireless Prototype
2021-05-09: Power Supply Module
2021-05-07: New Name
2021-05-06: Finished new UI
2021-05-04: Higher Bandwidth, new UI
2021-04-30: PCB Time
2021-04-29: Soldering the Processing Units
2021-04-28: Going Wireless
2021-04-24: First Amplifier Circuit
2021-04-14: Data Cleaning
2021-04-13: Cyber Gauntlet +1
2021-04-11: Adding some AI
2021-04-08: Baby Steps
2021-04-03: The Idea
INA155 Instrumentation Amplifier
I tried replacing the INA128 chips on the Electrode Module 3 with INA155 chips with a Gain of 50, and it works just fine :) The signal even looks a little bit cleaner than with the INA128, though I don't have good metrics to decide which one is better overall. They both have the same PCB footprint so I didn't even have to change the board design :)
This is good news. If I buy the INA155 chips in bulk (250+), they would cost ~1.75€ each (and I currently pay ~5.65€ per INA128), which would bring the material cost of the whole product down from ~60€ to ~30€ plus Arduino (~38€) plus shipment. Even if the INA155 is slightly worse, that's totally worth it.
At this price, it would even be affordable to use more than 8 signals (with additional analog multiplexing, since the Arduino only has 8 analog inputs) and wear more than two Electrode Module 3 at the same time.
Another big plus is that INA155 chips can easily work with a supply voltage of 3.3V!! And I wouldn't need to boost the battery voltage all the way up to 5V anymore, which is probably not very power efficient.
I did some more tests and concluded that with the electrodes that I'm currently using, the maximum gain of INA155 is not high enough, and I have to either find better electrodes, add extra amplifiers, or switch back to INA128 for the time being. For now, the relatively expensive INA128s will have to do.
But hey, the current electrodes are literally just... plain metal spacer screws, so there is DEFINITELY room for improvement.
Here are some pictures of electrode placement and resulting signals, where you can see that the INA128 (blue line) shows considerably more features:❬2021-12-15❭
Power Module 4
Unfortunately the large dimensions of the Arduino and the battery make it a little difficult. There's several options to do this, but none of them are perfect:
- Place Arduino and battery beside each other like in Power Module 3
- make the board ~15mm longer to fit the holes.
- Downside: The rubber band holes would be wider apart than on the Electrode Module 3
- make the board ~12mm wider to fit the holes.
- Downside: It would be too wide to comfortably fit on the radius bone
- Downside of both options: it might feel unbalanced when worn on the side of the arm, because the heavy battery moves the center of mass off to the side
- make the board ~15mm longer to fit the holes.
- Place Arduino and battery on top of each other, with pin sockets for the arduino that are taller than the battery, then make the board the size of Electrode Module 3 to fit the holes
- Downside: It might be hard to acquire pin sockets that tall
- Downside: Little space left for the holes and the remaining components, but might work
- Place Arduino and battery in a line next to each other and make the board 20mm longer and 8mm wider than Electrode Module 3 to fit the holes.
- Downside: The rubber bands are relatively close to the center, reducing the force with which the far ends of the board are pressed onto the skin
- Downside: The large dimensions might make it too bulky
- Remove the Arduino and solder the NINA-B306 bluetooth module of the Arduino onto the board directly, along with all the peripherals I need
- Downside: Unnecessary complexity
- Downside: More difficult soldering
- Instead of adding any holes, just stack the board on top of a Electrode Module 3 and use its screws and rubber band holes for physical support. (Thanks to PerlinWarp for that idea!)
- Downside: Would require a redesign of Electrode Module 3
- Downside: The height of the prototype might make it feel unbalanced
- Downside: The board would have be either wider or longer than the Electrode Module 3 due to the dimensions of the Arduino and battery. As a result, it would stick out to the sides, unless I enlarge the electrode board as well.
In the end I chose option 3 for simplicity reasons, and because the downsides seemed the most acceptable. This also gives me a lot of space on the PCB which is currently not needed, but may be used for analog multiplexing later on, to allow supporting more than 8 electrodes.
I am quite tempted by option 2 though, it would make for a really compact board. Maybe I give it a try later (if I ever find a source for 1x15 pin sockets that are 14mm+ tall and 2.54mm spaced apart.)
Here is the resulting board. As you see, I also added a screw mounting hole in the center, which should point right at the bone, which perhaps makes for a great ground electrode. Any of the screws can be used as ground electrodes though, simply by closing their respective solder jumper. I will try out various combinations to see which one is best.
The matching circuit:
I ordered the production of this board already, and I'm excited to see if it will work out like I imagine. :)
Fun story on the side: I ordered the boards on Aisler, and they let you change the PCB design for free until it hits production. This time, I had a couple of hours left, which I used to make a couple of adjustments here and there, until I was happy and went to sleep. After 3 hours I woke up, with racing thoughts, and I couldn't really fall asleep again (which happens very rarely). After laying awake for 2 hours, it occurred to me that I should run the automatic PCB layout check of KiCad once again to test my board for obvious mistakes. Of course I had run the check before submitting my order, and several times after doing my various modifications. But as it turned out, I had indeed forgotten to run it on the final version, and there were multiple errors too! Several unconnected nodes and some copper tracks that were too close to each other. Luckily I caught this just a couple minutes before my time for free modification ran out, and fixed it just in time. Thanks insomnia.❬2021-11-30❭
I haven't been posting recently because I've been focusing on other areas of my life, but there has still been progress with PsyLink:
Prototype 6 improvements
I redesigned Power Module 3 to be more easily hand-solderable by enlarging the solder pads and adding thermal clearances to some of the filled copper areas. (The copper tracks to several pads were so wide that the heat of the solder iron was dissipating too fast into the copper, and the pads were not heating up properly). The result is Power Module 3.1.
PerlinWarp reached out to me, and after some fruitful conversations, I build a Prototype 7 for him for research purposes. It's a hybrid of Prototype 4 and Prototype 6 which combines the advanced Power Module 3 with a cluster of Electrode Module 1 attached to Sleeve 3.
The assembly of the Prototype 7 took a full 20 hours of mindless labor, so I had a LOT of time to think about improvements to the design that would make it easier to assemble:
- Instead of having small boards for differential amplifiers that have to be tediously connected with three fragile wires, bulk four of them up in a single board.
- Instead of having a piece of custom-tailored fabric with precisely placed electrodes, use electrodes made of metal screw-like parts that fit right on to the PCB. The PCBs are held together with a rubber band that presses the electrodes tightly to the skin.
- Made Power Module 3 more easily hand-solderable (described above)
Prototype 8 (WIP)
The 8th prototype implements those changes, resulting in a device purely made of PCBs connected by rubber bands and wires. These PCBs rest on the skin on legs of metal, which double as the electrodes.
As of now, only the differential amplifiers are finished. The circuit for these is Circuit 10 (electrodes):
The front-side of the Electrode Module 3:
The four wide holes in the board are connection points for rubber bands, and the eight round metal-plated holes are mounting points the electrodes. These consist of a screw on one side, a nut on the other, holding both metal pieces tightly in place.
The electrodes can be spacer screws like these or dome nuts like these. While the spacer screws seem to pick up a better signal, the dome nuts are considerably more comfortable thanks to the rounded cap, so I think I'll stick with these for now.
Dome nuts are also called "hutmutter" in German, which translates to "hut's mother" (my nickname is hut), so just because of that fact, I had to try them out.
Additionally, one can use the pins EX1 through EX8 to connect arbitrary external electrodes.
I played around with using off-the-shelf wet EMS (electric muscle stimulation) electrodes for the ground electrode, since good connectivity for the ground electrode is important for a good signal.
I did not find an improvement in the signal though, perhaps wet electrodes are not worth it.
Finally, the global microchip shortage has affected this project as well, and I could not find any sources for the power converter chips (TPS61220 DCKR) that boost the battery voltage of 1.2V to the required 5.0V. I still have some legit chips left, but in the long term, I need more of these, so I acquired some chips from Shenzhen via AliExpress. They have been advertised as "Original new, 100% quality", so what could POSSIBLY go wrong? But I tried them out, and they actually do their job quite nicely.
The power module is currently only clumsily attached to the rubber band. The finished Prototype 8 will need a power module that's more compatible with the Electrode Module 3, by making it as long as the Electrode Module 3, and either by adding rubber band mounting holes, or by adding pin headers so that it can be stacked on top of the amplifier board, which would reduce the number of dangling wires as well.
I will also try out cheaper differential amplifiers to reduce the total cost of the device from ~70€ to ~30€ (excluding the ~39€ for the Arduino Nano 33 BLE Sense).❬2021-07-17❭
Neurofeedback: Training in both directions
For now, the training of neural networks mostly happens on the AI side. The human makes arm gestures and presses keys on the keyboard. This provides the input (electrode and IMU signals) as well as the output/labels (keyboard actions) to the artificial neural network, which then learns the correlation between the two through supervised gradient descent.
Now I'm looking into how to train both the neural network of the AI as well as the nervous system of the user through neurofeedback, that is, by making the user more aware of their neural signals, which in turn allows them to fine-tune these.
My hope is that this will make up for the low quality of information that's available to the AI, due to noise, attenuation, and the low number/quality of electrodes. The user neither knows what signals the electrodes can access, nor how to willingly produce movements that create these signals. Some gestures work well, while others can't be detected at all, so the best bet is to use forceful gestures with maximal muscle activation. But if there was some sort of feedback to the user, like a visualization of the data that the neural network is extracting, the user could focus on the movements that work, and gradually lower the intensity, perhaps to the point where no actual movement is required anymore.
Of course there is already some feedback about the signals: The PsyLink UI shows the amplitude of each electrode in a rolling graph, and the GNURadio application shows detailed plots of the raw signals, both of which already help determining which movements will work for gestures and which will not. But the AI can of course combine, cross-correlate, filter, convolve and deconvolve the signals, which enables it to extract information that a human won't see in the raw signal data.
Ultimately, the goal is that the user learns to, on demand, fire off just enough neurons that PsyLink can pick up the signal and trigger the intended key press without any visible movement of the arm.
As described above, simply presenting the user with the raw electrode data is insufficient. A machine-learning approach will likely be optimal here, to overcome the preconceptions of a top-down designer. Since we already have an artificial neural network, why not use that one to generate the visualizations too?
In my current version of this idea:
- The user needs to invent some arbitrary gesture that should correspond to the action "Press key 'A'".
- The user is repeatedly asked to perform the gesture by the UI
- At random intervals
- For random durations
- With 2-3 seconds of heads-up warning to account for reaction time
- In between the gestures, the user should perform random other activities, but never do the gesture without being asked by the UI
- The AI is trained on the fly with
- Electrode signals as input
- A binary label of "Key 'A' pressed" vs. "Key 'A' not pressed" as output
- Each data point is added randomly (80:20) to the training or validation dataset
- After X seconds of collecting data, the AI is trained for Y epochs
- Every Z milliseconds, the AI is asked to predict the output from the current
input, and the neural activations of the last non-output layer of the NN
are presented to the user visually, along with the predicted output.
- The visualization could be a heatmap or a scatterplot, for example
- The visualization should cover a large dynamic range (both small changes and large changes to the values should be easily visible)
- Using the feedback, the user can tweak their gesture as desired, to e.g.
- Minimize the movement required to trigger the key
- Maximize the reliability with which the key press is predicted
- Over time, old data is dropped from the NN training to refine the visualization and to keep the training time short.
Once the user is ready, they can add a second action like "Press key 'B'" and so on.❬2021-07-06❭
New Frontpage + Logo
The front page now looks a little more "modern", and I changed the logo from
Once again, this was inspired by the System Shock 2 Trioptimum logo.❬2021-06-24❭
Cyber Wristband of Telepathy +2 [UNIQUE ITEM]
Under the hood there's 9 electrodes (1x ground, 8x signal):❬2021-06-21❭
Running on AAA battery
Wow, it feels like ages since I started working on making PsyLink run on a rechargeable AAA battery. It sounds so simple and straight-forward, but it wasn't :). This ate 2.5 weeks of my time, but finally I succeeded!
Coming from software engineering, the iteration time of hardware prototypes is horribly slow. There was a lot of waiting for package deliveries, a lot of time assembling, and a lot of wrestling with leaky abstractions. For example, it wasn't enough to just connect the 1.2V->5V boost converter like on the circuit diagram, but I had to take special care of the distances between the parts, and the widths of the copper tracks connecting them.
Another problem you never face in software engineering is that the package with electronics parts was stolen, and when the new PCB of Power Module 3 arrived, I had to work with the few remaining (suboptimal) parts that I had.
What almost drove me insane was that I had only one fresh TPS61220 chip left. That's the 5V boost converter at position U1 (in the bottom of the red circle on the photo below), which is so small that I don't really have the tools to solder it on properly. I kept accidentally connecting the pins of the chip with solder. I gave up and started asking around friends for whether they could solder it on for me, when I remembered this soldering tip someone gave me: If you don't have soldering flux (which I didn't have), just use ✨margarine✨ instead ;D. It sounds very wrong, but it actually made a huge difference, and only thanks to the power of margarine I was able to keep the solder exactly where I needed it to be. "If it's stupid and it works, it's not stupid."
Then - for whatever reason - the voltage was going up to 1.5V instead of 5V, just like with the previous PCB layout. I thought I fixed that problem by optimizing the layout around the TPS61220 chip, but apparently that wasn't enough. I figured the 4.7µH inductance at L1 wasn't big enough, so I squeezed in a second coil:
(It looks like two tardigrades playing ball :) I wish I made a better picture before I disassembled it again)
This alone didn't help, but when I manually held an additional 10µH inductance in parallel to L1 for 1 second while the device was running, it surprisingly kickstarted and reached 5V!... until I drained some current, which made the voltage collapse back to 1.5V immediately. Curious behavior. :D
Then I tried soldering on the big 10µH coil directly onto the SMD pads, and the voltage reached 5V and stayed at 5V =)
No idea why the big 10µH coil worked while the two smaller coils totaling 9.4µH didn't work... I doubt that the 0.6µH difference in total inductance turned the tide, probably there's some factor I'm not aware of. I actually had ordered a 10µH SMD inductance coil in anticipation of this, but well, it got stolen... My only consolation is the face of the package thieves when they open the package, they realize that it's just a couple of tiny SMD parts, and they wonder WTF this shit is even good for.
P.S.: I found out that the reason why the coils didn't work was that they were not rated for the >200mA that's passing through them. Once I got a 4.7µH coil rated for 280mA, everything was fine.
The lack of parts forced me to adjust the resistance/inductance/capacitance values of Circuit 9 (power):
Thankfully I had planned for way too many capacitors on the PCB, just for some extra VROOOM, so the fact that I only had 10µF capacitors instead of the planned 100µF didn't matter too much.
I also changed the 1MΩ resistors in the reference voltage generator to 110kΩ because I read that smaller resistances in a voltage divider make the output voltage more stable, at the cost of more power use, but I think we can sacrifice some power for signal accuracy. No idea whether it's actually going to help though. I chose 110kΩ instead of the more common 100kΩ and 220kΩ because the boost converter already requires a 110kΩ resistor, and that way this prototype requires fewer different parts. But the exact values don't matter, as long as both resistances are identical.
Tomorrow I'll start working on the electrodes and wristband. :)❬2021-06-16❭
Power Module 3
It was still a worthwhile learning experience to build Power Module 2, resulting in the following changes:
- Instead of stacking the Arduino and the Battery on top of each other (one on the front side of the board, one on the back), they're now side by side, to make the board flatter and fit better on the forearm.
- Added more capacitors all over the place
- Added various jumpers:
- A solder jumper to enable/disable the reference voltage generator, in case I come up with a setup where I won't actually need it
- A solder jumper to sacrifice one electrode and use its analog input pin to measure the battery charge instead.
- Three configuration pins, connectable via jumpers, to change software settings on the fly and toggle between up to 4 different modes. Or you can use them to plug in an extra module.
- Hopefully fixed the layout for the step-up converter
Here's the circuit:
And the new PCB:
If you're wondering why the sparky, fancy looking power line goes all the way from the power switch on the right through digital pin D10 and into L1 on the left side... Indeed that looks pretty awkward, but L1 is the noisiest component, and I wanted to keep it as far as possible from the analog pins at the bottom, without sacrificing the effectiveness of the boost converter layout. Given the size & time constraints, I didn't find a better solution.
What time constraints, you ask? Well, actually I made a different PCB layout first. Polished every detail, and when it was perfect (according to my crude appraisal), I ordered it. Was already excited about the delivery, started putting the board on the website, and so on. But at some point I noticed that something was wrong... The Arduino pins were inverted. Theoretically, everything would still work, but the Arduino would have to be plugged in from the back side, which is something I wanted to avoid to keep the board laying nice and flat on the forearm...
Thankfully the manufacturing process hasn't started yet, and I could update the board for free. So I started redesigning half of the board and finished just in time for the production to start :D
Let's hope it works this time.❬2021-06-10❭
Believe The Datasheet
Today the order of Power Module 2 arrived!
And with relief I saw that the battery clips fit nicely onto the board, as does the Arduino (with pin strips), and all the other components. Just that simple thing already felt like an accomplishment at my level of expertise with PCB design ;)
After some dreadful time trying to solder on the tiny 1x2mm-sized chip at U1 (I need a microscope for this shit), I had the SMD parts assembled and the circuit was ready for a test drive:
But something was weird. The output voltage was a meager 1.5V, not the expected 5V, even though everything was connected properly. After hours of debugging I flipped over the table and just soldered a fresh board, this time without the Vref-generating OpAmp (U2+R3+R4+C3). But no luck, still just 1.5V. This was lower than the minimal output voltage of the voltage booster, so the chip didn't even finish it's start-up phase. How could that be, if there's not even any load on the output voltage?
I desperately tried several different things. One was doubling the inductance at L1 from 4.7µH to 9.4µH by using two SMD inductances in series:
Unfortunately I think I broke the coils while constructing this, since they didn't let any current go through. But I found a regular, big inductance coil with 10µH, manually held its pins down onto the SMD pads, and indeed, the voltage jumped up to 5V!
So I soldered it... onto... the SMD pads.
(Probably you can reconstruct my entire room from all the reflections in this image, along with a biometric picture of my face and at least 3 of my fingerprints...)
But once I connected the OpAmp, the voltage went back down to 2.5V, and with the Arduino connected, it went down further to 2.4V. Adding a second coil in series for a total of 20µH didn't compensate for this, but made it even worse, bringing it all the way down to 1.5V.
Well, clearly whatever is wrong with this construction has something to do with the inductance, and it's not purely the amount of inductance... Which brings us to the title of this post:
The mistake (probably)
Of course the data sheet of the voltage booster CLEARLY STATED that the inductance coil needs to be AS CLOSE AS PHYSICALLY POSSIBLE to the chip. The capacitors C1 and C2 too, by the way. And I even read that. But I thought, what could possibly go wrong if I move it ~1cm away to make some space for the battery? Nothing, right? Well, awesome, I guess it's time for another revision :)
At least the 2.4V were enough to power the Arduino, although it was visibly struggling. I could establish a Bluetooth connection and collect some signals, but the Bluetooth packets were coming in extremely slowly (though still faster than mobile internet in 80% of Germany.)
Somehow the oversized inductance coil adds a nice vibe to it.
Front, with battery:
Back, with attached Arduino:
This picture shows a pin strip socket that will be gone in the final version, where the Arduino will be soldered onto the board, reducing the height from 3cm to 2cm.
Side, with Arduino and battery:
(Yes, the pin strip socket is too long by one pin ;))❬2021-06-04❭
Back to the Roots
While uploading the old videos to the new PeerTube channel, I viewed the first video once again, which shows a pretty good signal from just two pieces of aluminum foil taped to the skin. And I wondered, why do I even bother with such a complicated set-up like in Prototype 4? It was really annoying to assemble, and the device is clunky and fragile.
Let's go back to the roots and build something more simple. Plenty of reasons:
- Having 8 small, distributed signal processing PCBs sounds nice at first, but it creates more points of failure. With Prototype 4, I had multiple instances where a connecting wire broke off, so this time I'd rather have everything on one PCB.
- What happened to the idea of processing the signal as little as possible and leaving the job to the neural network? Circuit 6 (whole device) amplifies certain information but hides other, like the DC voltage offset between electrodes.
- The electrode map of Prototype 4 provides me with signals (like "arm
turns left/right") which are redundant since I added support for the
inertial measurement unit (IMU).
- The signal from the IMU is actually way more reliable than what I'm getting from the electrodes, so why not ditch them completely? Well, there are some parts of the arm that move separately from the Arduino, mainly the fingers. So let's focus on these.
- My main issue with the early prototypes was the poor signal-to-noise ratio, but that was mainly due to the power line. This should be gone with a battery-powered device.
The Circuit 7 (whole device) shows a simplification of the signal processing module to a simple non-inverting amplifier per electrode with a gain of 221x. There's also a 560KΩ bias resistor towards Vref so the voltage we measure isn't too far off the center. In Circuit 6 (whole device) I had used 1MΩ, but here it produced mysterious oscillations, and going down to 560KΩ mysteriously fixed it.
This circuit also features a rechargeable 1.2V AAA battery with a TPS61220 step-up converter boosting the voltage to 5V, because I don't hate nature, and I burned through enough CR2032 coin cells. Coin cells also aren't exactly optimized for currents of 20mA, and thus get drained too quickly.
I measured a signal while pressing down a finger onto the table with two electrodes along the Flexor Digitorum Superficialis. Blue is electrode 1, red is electrode 2, and green is an amplified difference:
But let's cut even more out of this circuit. Here's one that is designed to be a shield to the Arduino Nano 33 BLE Sense, containing just the power supply, and an array of pass-through pins:
The signal that I'm getting is weaker, but certainly usable: (again, I measured a signal while pressing down a finger onto the table with two electrodes along the Flexor Digitorum Superficialis. Blue is electrode 1, red is electrode 2, and green is an amplified difference)
I also built a PCB that implements this power supply/pass-through shield, and I figured, even if the device ends up not very useful, I'll still be able to use this for experiments later on, thanks to the pass-through pins.
I just hope that the PCB/circuit will work at all. I still haven't figured out how to simulate it, and I don't really know the best practices for PCB design. The PCB footprint for the AAA battery clips (Keystone 82) is my first custom-made PCB footprint too. Hope it all works out.
It will be a relatively small forearm band with 8 electrodes (+ 1 ground electrode), which I plan to place around the Flexor Digitorum Superficialis for detecting what individual fingers are doing. The information from the gyroscope + accelerometer should cover the rest.