PsyLink
❬2021-04-24❭

First Amplifier Circuit

I had my head stuck in electronics lectures, datasheets, and a breadboard to figure out a decent analog circuit for amplifying the signal. It sounds so straight forward, just plug the wires into the + and - pin of an operational amplifier, add a few resistors to specify the gain of the OpAmp, and feed the output to the analog input pin of the Arduino... But reality is messy, and it didn't quite work out like that.

Here's a list of problems:

I also connected the electrode signal to ground with a 1MΩ resistor which greatly improved the signal, and I have no idea why.

One peculiar thing I noticed was that the signal seemed stronger when my laptop was connected to the power supply. It superimposed noise, but also seemed to increase differences in electrode voltages. I don't quite understand this yet, but 2 things follow from that:

Some of the references I used:

The resulting circuit:

Circuit schematic

And the signals look like:

Signal image 1 Signal image 2

Yellow and green are two electrodes, right after their respective OpAmp, and purple is (yellow-green)*20.

This should be good enough to move forward, but I bought some INA128 instrumentation amplifiers and perhaps I will tinker some more to get an even better signal. Can't wait for the next prototype though :).

In other news, I watched Dr. Gregory House explain forearm muscles, so next time my electrode placement will be better than random!

And since I learned KiCad for creating the above schematic, I thought I'd add schematics for the previous models as well, see circuits.

❬2021-04-19❭

Amplifiers

I have the feeling that before building the next prototype, I should figure out some way of enhancing the signal in hardware before passing it to the microcontroller. It's fun to hook the 'trodes straight to the ADC and still get results, but I don't think the results are optimal. So these days I'm mostly researching and tinkering with OpAmps.

❬2021-04-15❭

Multiplexers

The analog multipexers (5x DG409DJZ) and other stuff arrived! I almost bought a digital multiplexer, because I didn't know there were various types... But I think that these will work for my use case. The raw signal that I get out of it looks a little different, but when I filter out the low & high frequencies with TestMultiplexer2.ino, the direct signal and the one that goes through the multiplexer looks almost identical =D.

❬2021-04-14❭

Data Cleaning

The arduino code now produces samples at a consistent 1kHz. I also moved the serial read operations of the calibrator software into a separate thread so that it doesn't slow down on heavy load, causing the buffer to fill up, and the labeling to desynchronize. I am once again confused and surprised that I got ANY useful results before.

I disconnected analog input pin 7 from any electrode, and used it as a baseline for the other analog reads. By subtracting pin 7 from every other pin, the noise that all reads had in common was cancelled out. Hope this doesn't do more harm than good.

I also connected the ground line to one of the wrist electrodes rather than to the palm, since the palm electrode tended to move around a bit, rendering all the other signals unstable.

And did you know that the signals looks much cleaner when you unplug the laptop from the power grid? :p

screenshot

I'll finish with a video of me trying to play the frustrating one-button jumping game Sienna by flipping my wrist. This doesn't go so well, but maybe this game isn't the best benchmark :D My short-term goal is to finish level 1 of this game with my device.

❬2021-04-13❭

Cyber Gauntlet +1

So if you ever worked with electromyography, this will come to no surprise to you, but OMG, my signal got so much better once I added a ground electrode and connected it to the ground pin of the Arduino. I tried using a ground electrode before, but connected it to AREF instead of GND, which had no effect, so I prioritized other branches of pareto improvement.

I am once again confused and surprised that I got ANY useful results before.

For prototype #3, I moved the electodes further down towards the wrist in hope that I'll be able to track individual finger movements. It had 17 electrodes, 2x8 going around the wrist, as well as a ground electrode at the lower palm. Only 9 of the 17 electrodes are connected, 8 directly to the ADC pins, and one to 1.65V, which I created through a voltage divider using two 560kΩ resistors between the 3.3V and GND pins of the Arduino, so that the electrode signals will nicely oscillate around the middle of the input voltage range.

It all started out like a piece of goth armwear:

photo

Photo from the testing period:

photo

Soldering wires to the electrodes:

photo

The "opened" state shows the components of the device:

photo

But it can be covered by wrapping around a layer of cloth, turning it into an inconspicuous fingerless glove:

photo

If you look hard at this picture, you can see the LED of the Arduino glowing through the fabric, the voltage divider to the right of it, appearing like a line pressing through the fabric, the ground electrode on the lower right edge of my palm, and the food crumbs on my laptop :)

The signal seems to be much better, and as I move my arm and hand around, I can see distinct patterns using the Arduino IDE signal plotter, but for some reason the neural network doesn't seem to process it as well. Will need some tinkering. I hope it was not a mistake to leave out the electrodes at the upper forearm.

I already ordered parts for the next prototype. If all goes well, it's going to have 33 'trodes using analog multiplexers. The electrodes will be more professional & comfortable as well. Can't wait!

❬2021-04-11❭

Adding some AI

Most neural interfaces I've seen so far require the human to train how to use the machine. Learn unintuitive rules like "Contract muscle X to perform action Y", and so on. But why can't we just stick a bunch of artificial neurons on top the human's biological neural network, and make the computer train them for us?

While we're at it, why not replace the entire signal processing code by a bunch of more artificial neurons? Surely a NN can figure out to do a bandpass filter and moving averages, and hopefully come up with something more advanced than that. The more I pretend that I know anything about signal processing, the worse this thing is going to get, so let's just leave it to the AI overlords.

The Arduino Part

The Arduino Nano 33 BLE Sense supports TensorFlow Lite, so I was eager to move the neural network prediction code onto the microcontroller, but that would slow down the development, so for now I just did it all on my laptop.

The Arduino code now just passes through the value of the analog pins to the serial port.

Calibrating with a neural network

For this, I built a simple user interface, mostly an empty window with a menu to select actions, and a key grabber. (source code)

The idea is to correlate hand/arm movements with keys that should be pressed when you perform those hand/arm movements. To train the AI to understand you, perform the following calibration steps:

  1. Put on the device and jack it into your laptop
  2. Start the Calibrator
  3. Select the action "Start/Resume Recording" to start gathering training data for the neural network
  4. Now for as long as you're comfortable (30 seconds worked for me), move your hand around a bit. Hold it in various neutral positions, as well as positions which should produce a certain action. Press the key on your laptop whenever you intend your hand movement to produce that key press. (e.g. wave to the left, and hold the left arrow key on the laptop at the same time) The better you do this, the better the neural network will understand wtf you want from it.
    • Holding two keys at the same time is theoretically supported, but I used TKinter which has an unreliable key grabbing mechanism. Better stick to single keys for now.
    • Tip: The electric signals change when you hold a position for a couple seconds. If you want the neural network to take this into account, hold the positions for a while during recording.
  5. Press Esc to stop recording
  6. Save the recordings, if desired
  7. Select the action "Train AI", and watch the console output. It will train it for 100 epochs by default. If you're not happy with the result yet, you can repeat this step until you are.
  8. Save the AI model, if desired
  9. Select the action "Activate AI". If everything worked out, the AI overlord will now try to recognize the input patterns with which you associated certain key presses, and press the keys for you. =D

Results

I used this to walk left and right in 2020game.io and it worked pretty well. With zero manual signal processing and zero manual calibration! The mathemagical incantations just do it for me. This is awesome!

Some quick facts:

Video demo:

Still a lot of work to do, but I'm happy with the software for now. Will tweak the hardware next.

Now I'm wondering whether I'm just picking low hanging fruits here, or if non-invasive neural interfaces are really just that easy. How could CTRL-Labs sell their wristband to Facebook for $500,000,000-$1,000,000,000? Was it one of those scams where decision-makers were hypnotized by buzzwords and screamed "Shut up and take my money"? Or do they really have some secret sauce that sets them apart? Well, I'll keep tinkering. Just imagine what this is going to look like a few posts down the line!

❬2021-04-09❭

F-Zero

The look of the first device was way too unprofessional, so I pulled out my sewing machine and made a custom tailored sleeve from comfortable modal fabric.

photo

On the inside, I attached some recycled studs that served as electrodes. Who needs that expensive stuff they sell as electrodes when a piece of iron suffices?

photo

This time it had 4 electrodes. I targeted the middle and the distal end of two muscles, the Brachioradialis and the Extensor carpi radialis longus. I picked those muscles at random, because I honestly don't know what the fuck I am doing.

Software-wise, I played around with moving average and got reasonable signals, but it was clear that there was too much noise.

How to filter, though? I'm not going to solder some bandpass filter, that's too slow and inflexible. There are simple algorithms for doing it in software (link 1, link 2), but something seemed off about this method. In the end, I decided to learn how to do a Fourier transform on the Arduino.

With this code (inspired by this post), I took 64 samples at a sampling rate of 1kHz, performed the Fourier transform, cut out anything under 30% and over 50% of my frequency range, and then summed up the amplitudes of the remaining frequencies to generate the output.

Still very crude, but it allowed me to get distinctive signal patterns for various positions of my arm:

screenshot

I was genuinely surprised that I got information of this fidelity and usefulness from just hooking up 4 ADC's to semi random places of my forearm and a software bandpass filter. This was good enough to use it as a basic input device!

I wondered, can I control a racing car game with this?

To test that, I built this program to read out the signals and convert certain ranges of values to keyboard presses of the keys Left and Right. The value ranges need to be calibrated before each use: I held my left arm like I'm grabbing an invisible steering wheel, moved it left and right, and looked hard at the signal values to find correlations like "signal A is always below X if and only if I steer left". Once the calibration is done, the invisible steering wheel turned into a magical keyboard with 2 keys =D

Right away I tried it out to steer in my favorite racing game, F-Zero:

Note that in addition to the steering wheel, I used my other hand to accelerate.

I loved it, but there is still a lot of work to be done. The calibration is a pain, especially since it needs to be repeated if the electrodes move too much, which happens a lot with this kind of sleeve. Also I want more electrodes, better signal processing, and data transfer via Bluetooth so I can run it off a battery.

❬2021-04-08❭

Baby Steps

The Arduino arrived. I have no electrodes though. But what are electrodes, just some pieces of metal taped to your skin, right? Let's improvise that:

photo

There are two pieces of aluminum foil taped to my skin, held together with blue medical wrap.

The educational material about electromyographs that I've seen described a chain of hardware elements to process and clean up the signal:

But I thought, let's focus on the MVP. Why not simply hook the electrodes straight to the analog input pins of the Arduino with some alligator clamps? Worked fine. I did minimal signal processing in software though, you can find the source code here.

This video shows the myoelectric signal on Arduino IDE's built-in signal plotter:

❬2021-04-03❭

The Idea

On this day, I got the idea and started researching EMG design and signal processing, motor neurology basics, as well as existing projects.

Soon I realized that I will need a microcontroller to record and process the signals. I considered the Raspberry Pi Pico and Arduino Nano 33 BLE Sense, and chose the Arduino because:

I wish there was a decent battery/UPS shield, couldn't find one so far.

Page: 1 2 3 4 all