diff options
Diffstat (limited to 'src/modelli/lego-piano/document.en.rest.txt')
| -rw-r--r-- | src/modelli/lego-piano/document.en.rest.txt | 258 |
1 files changed, 258 insertions, 0 deletions
diff --git a/src/modelli/lego-piano/document.en.rest.txt b/src/modelli/lego-piano/document.en.rest.txt new file mode 100644 index 0000000..309d5c6 --- /dev/null +++ b/src/modelli/lego-piano/document.en.rest.txt @@ -0,0 +1,258 @@ +====================================== +Making the LEGO Grand Piano play music +====================================== +:CreationDate: 2020-08-28 10:39:28 +:Id: modelli/lego-piano +:tags: - 3dPrint + - hardware + - software + - models + +The `LEGO Grand Piano 21323 +<https://www.lego.com/en-gb/product/grand-piano-21323>`_ is a great +model, full of clever details and impressive engineering. + +It contains a bit of electronics, that allow it to talk with a phone +app via Bluetooth. + +.. image:: lego-electronics.jpg + :width: 100% + :alt: a Lego optical sensor, motor, power&control brick, mounted on + a wide base of sideways pieces; the motor moves a long axle + with many pegs at right angles to each other + +The app can play a few built-in tunes while running the motor inside +the piano, which pulls some keys down in a regular pattern, so you can +pretend to have a "player piano". The app can also wait for you to +press any key on the piano (they all move a little flag in front of +the optical sensor) before playing the next note, so you can pretend +to be playing. + +That's fun, but not really enough: there are 25 independent keys, +surely we can make them play the right note! + +.. image:: keys.jpg + :width: 100% + :alt: the full keyboard of the Lego piano, 2 octaves + 1, with + correctly-alternating white and black keys, each key has a + matching hammer + +.. video:: keys-hammers.mp4 + :width: 100% + :type: video/mp4 + +After some research, I decided to use optical sensors and a big +microcontroller. How hard can it be? ☺ + +As sensor I picked the `QRD1114 +<https://learn.sparkfun.com/tutorials/qrd1114-optical-detector-hookup-guide/all>`_, +which is dead easy to use: infrared LED plus phototransistor, with a +very narrow and short area of sensitivity, so there's very little +chance of nearby keys setting off the wrong sensor. + +The controller I used is a `Lilygo TTGO-T7 v1.3 +<http://www.lilygo.cn/prod_view.aspx?TypeId=50033&Id=1258&FId=t3:50033:3>`_, +just because I had bought a bunch of them. It's a very small ESP32 +board, with 40 pins broken out, and support for charging a lithium +battery from USB. + +.. note:: + + The v1.3 is a ``d1_mini32`` board as far as the `esp32-arduino + <https://github.com/espressif/arduino-esp32>`_ libraries are + concerned, v1.4 is a ``esp32wrover``. There are probably other + variants, so you may need to adjust something if you want to + replicate my build + + +First step was figuring out how to connect the sensors to the +controller. The LED in the QRD1114 can easily work with about 20mA, +so I can drive one directly from the ESP32, whose GPIO pins can drive +up to about 30mA. 25 sensors fit nicely in a 5×5 matrix, so something +like this should work: + +.. image:: lego-piano-scanner-schematic.svg + :width: 100% + :alt: schematic drawing of an electronic circuit; four QRD1114 are + arranged in a 2×2 grid; a pin labelled "row1" is connected to + 2 LED anodes via a 330Ω resistor and to 2 phototransistor + collectors via a 10kΩ resistor; a pin labelled "row2" is + similarly connected to the other pair; a pin labelled "col1" + is connected to the LED cathode and phototransistor emitter + of two sensors, one per "row"; a pin labelled "col2" is + similarly connected to the other pair; between each 10kΩ + resistor and the phototransistor collectors are pins labelled + "sense1" and "sense2" + +Pulling a "row" pin high (with the others low) and a "col" pin low +(with the others in high-impedance / tristate), we can turn on one +sensor at a time, so we can scan the matrix. When the phototransistor +sense light reflecting from an object in front of it, it will pull +down the "sense" pin, which we can read via the ADC in the controller. + +Testing that design on a breadboard proved that it can work! + +.. image:: sensor-matrix-test.jpg + :width: 100% + :alt: breadboard with a ESP32 dev board, 4 QRD1114 sensors, four + resistors, and a bunch of wires. One QRD1114's LED is shining + a weak pink, the others are off + +How are we going to keep the sensors in the right place inside the +piano, though? We're going to print a support that's compatible with +Lego pieces! + +.. image:: sensor-mount-test.jpg + :width: 100% + :alt: white plastic 3D-printed rectangle, 6×2 Lego studs in size, + with standard-sized studs only along one long side; 5 QRD1114 + sensors occupy one long side, and 2 Lego 1×1 plates with C + clip are on the other side + +My printer (a `Prusa i3 MK3S +<https://shop.prusa3d.com/en/3d-printers/180-original-prusa-i3-mk3s-kit.html>`_) +can print holes of the right size for standard vias, and can print the +whole sensor support in one go (29 studs long). The sensors need to be +aligned with the hammers (there's no space behind the stems of the +keys, also the hammers are white so more visible to the +phototransistor, and move more, so it's less probable we'll have false +positives). I used `OpenSCAD <http://www.openscad.org/>`_ to `model +the support +<https://www.thenautilus.net/cgit/lego-piano/tree/3d-print/qrd-holder.scad>`_. + +.. image:: sensor-mount-size-check.jpg + :width: 100% + :alt: white plastic 3D-printer rectangle, 29×2 Lego studs in size, + with 25 groups of 4 small holes to hold the sensors, placed + on top of the "strings" on the piano, showing how the holes + for the sensors align with the hammers + +We also must check positioning on the other two axes, of course. + +.. image:: sensor-position-test-1.jpg + :width: 100% + :alt: one QRD1114 mounted on the 3D-printer plastic holder, + suspended below one of the "strings" with a Lego clip piece; + the sensor sits just above and behind the rightmost hammer on + the keyboard; the hammer is in its resting position + +.. image:: sensor-position-test-2.jpg + :width: 100% + :alt: same pieces as before; the hammer is in its raised position, + precisely in front of the sensor + +At this point someone must be asking: how are we going to solder those +sensors on a *plastic* board? And the answer is, we're not going to! I +decided to go with `wire-wrapping +<https://en.wikipedia.org/wiki/Wire_wrap>`_! + +.. image:: sensors-mount-wiring.jpg + :width: 100% + :alt: reverse side of the 29×2 rectangle, with 10 sensors inserted + into their holes from the front, and many thin electrical + wires, insulated in yellow plastic, connecting some of their + pins + +The sensors have 4 pins, numbered counter-clockwise looking at the LED +/ phototransistor faces: + +1. phototransistor collector, to connect to pull-up resistor and to + "row" pin +2. phototransistor emitter, to connect to "column" pin +3. LED anode, to connect to current-limiter resistor and to "row" pin +4. LED cathode, to connect to "column" pin + +so the whole wiring looks like this:: + + aM aN aO aP aQ cM cN cO cP cQ + 14 14 14 14 14 14 14 14 14 14 … + 23 23 23 23 23 23 23 23 23 23 … + Mb Nb Ob Pb Qb Md Nd Od Pd Qd + +where ``a``, ``c`` go to the pull-up resistors; ``b``, ``d`` go to the +limiter resistors, and ``M``, ``N``, ``O``, ``P``, ``Q`` go to the +column pins. + +The controller board is built in a similar way: 3D-printed and +wire-wrapped (see `the model +<https://www.thenautilus.net/cgit/lego-piano/tree/3d-print/controller.scad>`_). + +.. image:: board.jpg + :width: 100% + :alt: white plastic 3D-printed rectangle, with two groups of 10x2 + holes around a label "esp", two groups of 5×2 holes around + labels "220Ω" and "10kΩ", three groups of 5 holes beside + labels "led", "gnd", "pht", and one group of 9 holes beside a + label "amp"; there is a notch in the rectangle near the "esp" + label, and a small raised block near the "amp" label + +The notch in the board corresponds to the battery connector, and the +raised block is to hold up the small `AdaFruit audio amplifier +<https://learn.adafruit.com/stereo-3-7w-class-d-audio-amplifier/inputs-and-outputs>`_ + +.. image:: board-populated.jpg + :width: 100% + :alt: same board, with all the components placed on it. The "led", + "gnd" and "pht" holes hold connectors + +.. image:: board-wired.jpg + :width: 100% + :alt: reverse of the populated board, many electrical wires + insulated in yellow plastic connect various pins + +.. image:: board-full.jpg + :width: 100% + :alt: same populated board, with a small naked speaker connected to + the amplifier + +I had some problems with the row / column connections, because not all the +GPIO pins can actually be used. After some trial and error, I settled +on: + +- row pins: 05 23 19 18 26 +- colums pins: 17 33 16 21 22 +- ADC pins: 02 04 12 27 14 +- DAC pin: 25 +- amp enable: 32 + +The program was a bit fiddly to get right, but not particularly +complicated, `you can see it in my Git repository +<https://www.thenautilus.net/cgit/lego-piano/tree/esp32/lego-piano.ino>`_. + +.. note:: + + The TTGO board gave me some problems with uploading the compiled + image, with errors like ``A fatal error occurred: Timed out waiting + for packet content`` or ``Invalid head of packet (0xE0)``. To fix + those, I had to `set the upload speed by hand + <https://www.thenautilus.net/cgit/lego-piano/tree/esp32/Makefile>`_ + in the ``Makefile``. + +First test with a few sensors on a breadboard: + +.. video:: sound-test-1.mp4 + :width: 100% + :type: video/mp4 + +and with all the sensors, mounted inside the piano: + +.. video:: sound-test-2.mp4 + :width: 100% + :type: video/mp4 + +I removed the Lego electronics to make space for the wires and the +controller board. + +And, finally, the whole assembled set: + +.. video:: sound-test-3.mp4 + :width: 100% + :type: video/mp4 + +At the moment the program can't really deal with more than one key +pressed at a time, as you may have noticed at the end of that last +video. The next step is to use a soundfont library, probably +`TinySoundFount <https://github.com/schellingb/TinySoundFont>`_ or `the +ESP-optimised version +<https://github.com/earlephilhower/ESP8266Audio/tree/master/src/libtinysoundfont>`_, +which should have no problems mixing multiple notes. |