Making Suwako Moriya’s Hat: Part 2

A continuation of the previous post, Making Suwako Moriya’s Hat: Part 1.

So it’s been a while since I posted about this, but now it’s done (and have been for a while), so let’s continue! If you only want to see a demo, there’s a video in the end.

After doing some basic stuff with coding and the 8×8 matrices, I took a little break while being busy with some other things. When I finally got some free time, I started on the other end, making the actual spheres that would be the eyes. I figured the easiest way would be to 3D print spheres.

My first try was to make a sphere, and have pipes going from the surface down to the LED matrix that would be placed in the middle of the sphere.

The first sphere I designed, with pipes to each LED

The first sphere I designed, with pipes to each LED

I used 123D Design to create a dome with 64 pipes to the edge. Was easy enough, just that I didn’t have any way to print it… Then a friend suggested 3DHubs to me, a site where you can find people near you that prints your models on their 3D printers for a fee. Unfortunately my sphere was big and the pipes used a lot of material, and the price was just too expensive. That’s when I got the idea to cut down the pipes to be shorter, but not too short, the matrix still has to fit inside. Doing that saved loads of material, and I could get it down to 180SEK ($21) for one dome at the cheapest place.

Unfortunately it was very hard to print right and still took a long time, and I quickly discovered that it didn’t really work as I had hoped, the pipes were leaking light, as can be seen in the video below. It’s also really really huge.

I decided to give up on the 8×8 matrix, the spheres were just too large and too expensive to print at higher detail. I decided to try with a 6×6 matrix instead, but I couldn’t find that dimension and figured I would have to create one myself. A friend of mine had suggested buying LEDs from Ali Express, and I bought loads of the same chip that’s used in NeoPixels for cheap, about $16 per 100 LEDs. These come in blocks of 10×10 LEDs, unless they break while shipping. Which they did. I ended up using superglue to glue LEDs back together to create a 6×6 matrix.

The only issue was soldering. I never soldered in my life, and I didn’t want to spend too much on tools. All I got was a cheap solder iron with adjustable temperature and inaccurate hands. It was extremely frustrating experience and took almost a week per matrix since I couldn’t stand working on it for too long (having to desolder several times, since the solder pads were really close to each other). I think I spent around 5 – 6 hours of soldering on each matrix. And since the solder pads were so close to each other, I couldn’t use straight wires since they would break however I tried, and had to bend them. I was really worried the soldering would break during me walking around.

From previous experience using white, thin pipes didn’t work very well. My next idea was to make a thin, white dome and a solid inner part made out of black material.

The inner part

The inner part

It was still quite expensive to make, but it worked a lot better than the old model. Unfortunately, the brightness on the surface wasn’t very great, it worked in a dark room but would probably be near invisible outside.

I started looking around for ideas, and that’s when I found the best thing possible, in the local hobby chain store. There was this clear Christmas ball that had an inner wall in the middle that could be removed. And amazingly, it had the exact same dimension as the sphere I made for the 6×6 matrix, meaning it would fit just perfectly!

Unfortunately, being in the middle of the summer, it wasn’t in stock. But they had it in their warehouse, and a week later I could go and collect them. I bought some white, acrylic spray paint and painted one side of the inner wall white, and the inside of one of the domes white, but prior to painting I drilled a small hole for the wires to enter the sphere. Placing the LED matrix on the wall and closing worked alright, but needed some effort. I didn’t have to use any glue or anything, since the wires would actually fit just enough into the dome that it would push the matrix against the wall. After some testing, it worked really great and it was extremely bright. Nice!

Well then. Now we have the LED matrices and the eyes. What about the hat? This was a bit harder.

I figured I would buy some thick paper in the same hobby store, but how would I cut it? Since Suwako’s hat isn’t really a top hat, I can’t just roll it up. Luckily, I figured if I made a cone of a specific size and cut that in half it would work. Said and done, after doing some calculation, cuttings and gluing, I had a Suwako hat. But the paper looks really ugly and doesn’t look as a proper hat at all. I got some fabric, but how would I cut the fabric and wrap it around the hat? Since I had already assembled the paper, I figured it would be a bit harder. And I wasn’t very happy with the hat either, as it was a bit smaller than I had hoped, since the papers dimension was a limiting factor (the store only had sheets of 64x46cm).

I went to a store that specializes in paper and paper products, and found a thick sheet of paper that were 100x70cm, perfect. But I wanted to do the perfect cuts too, and be able to visualize the hat before assembling. Back to 123D design we go!

Model of the hat

Model of the hat

I modeled up a hat with the dimensions I wanted and made hole for the eyes. But how do I make a template out of it? With Pepakura Designer, of course! Pepakura is a software to import 3D models and make paper crafts of them by unfolding the 3D model onto 2D. Usually I’d guess it wouldn’t be used on this kind of scale, but it’s worth a shot! Unfortunately, it didn’t like the STL’s that 123D Designer exported. There were tens of thousands of faces, and Pepakura warned for this. The resulting unfold was rather messy. I talked to a friend of mine on Steam that I makes paper craft and I’d figure also know about Pepakura. They looked at it for a bit and cleaned it up, and I got back the template. Thank you!

Obviously I didn’t have a printer that could print to a 100x70cm sheet, but luckily Pepakura can split a template up into several A4’s, that I assembled and cut out from the sheet, and cut out some bigger parts from the fabric. After some work, I had a hat assembled. Unfortunately I’m rather bad at creating stuff with my hands so the hat got a bit skewed but it wasn’t very noticeable. Inserting the eyes were quite hard to get right, and I had to have the matrix powered on to know the correct orientation. Unfortunately the hole wasn’t always oriented properly with the eye (using the arrow eyes for orientation), but that was easily fixed in code later, as long as the arrow pointed top, left, right or down and not in between. Unfortunately it was a bit hard and I didn’t manage to get it completely right, but it still turned out alright.

Then, back to the code! I had most things worked out, except for the Android app. And it was only a few days left for the convention. It was some hectic days, but after a while, it was all done!

Going to the convention was a weird experience, since I got there by taking a 4 hour bus. But to get to the bus I had to take the local tram to the larger bus station. Some people looked a bit confused at me, but no one really cared at all. Hat in action

It worked really well during the convention, but it was rather heavy and uncomfortable to wear, since the hat would press down and bend my ears at all time. Sadly, my fear of soldering slightly came true on the second day, when one of the eyes started acting up, where the colors wouldn’t be the proper color or the LEDs weren’t bright. A friend at the convention suspected it being a power issue, and might as well, but after accidentally putting the hat down a bit too hard, it started working again!

It was fun and I enjoyed making it, but I’ll probably make something that’s easier on my ears next time! Source code is still available on GitHub!

Complex data structures with node-ffi

I recently got tasked with creating a node.js wrapper around one of our C libraries at work. Since I already created a java wrapper around the same C library, I knew what I needed to do and started looking into some options, which were to either create a node addon or using node-ffi. I was tired of writing stuff in C (a language I’ve barely used) since I made the java wrapper in JNI, I chose to try node-ffi.

Node-ffi was really easy to get started with, as long as your library functions used primitive datatypes. However, our C library uses a lot of structs, and that’s where things got complicated. Fortunately, there is the ref-struct node module, which makes it easy to define a struct. However, we had some string maps and string arrays that consisted of a struct holding a pointer and a length, something like this:

On the javascript side, it would be mapped like this:

Easy enough? Here’s where I got stuck. The C library would return a struct with a length of 7 (len set to 7).

You can probably get the issue, how do I get all of the elements? It took me quite a while, but here’s how I did:

This will create a javascript object of the StringMap, using the keys as object keys.

Let’s check the documentation for ref.get(buffer, offset, type). It accepts the buffer holding the data, an offset and the type. What we have to do is to iterate over the pointers, in steps of 4 bytes on 32-bit systems and 8 bytes on 64-bit systems. Using ref.sizeof.pointer we can get the correct size. This can also be used on a string array made the same way (a struct containing a pointer to a string (char *) and a length), just skip the key and push the return value of ref.get to an array).

But what about a struct containing an array of a different struct? Here it becomes complicated.

And on javascript:

The language struct takes 24 bytes to store on a 64-bit system (3 fields * 8 bytes). If you try to use the same code as we did for our string map:

You soon notice it will throw an exception that the passed in buffer needs to be at least 24 bytes. I assume it refers to language, so I tried to pass in a buffer with a size of 24 or more, or do a ref.alloc(language) since the documentation said it would create a buffer with the correct size and the correct type set. However, that would throw an error “could not determine a proper “type” from: [0,0,0,0,0,0,0,0,32,2,75,1,0,0,3,0,91,44,93,23,0,0,0,0]“. Well, not really helpful. I tried reading the documentation and trying out various things, but nothing worked. Then by sheer luck, when I was ready to give up, I managed to stumble upon one of the fields:

What the hell? So turns out that when calling ref.get with the type of ref.types.CString, we can get the values of the strings. They seem to come in the same order as the struct is defined, so at offset 0, we get the id field of the first struct, at offset 8 (element 0 + size of CString on a 64-bit os) we get the name field of element 0, at offset 16 we get the code field of element 0, at offset 24 we get the id field of element 1 and so on and so on. In this snippet, 3 is the number of fields in the struct, and is valid if each field is a string. If a field is something else than a string, you obviously need to make changes to the offset calculations and the size parameter (the last one).

This means you have to recreate your data structure and this may not be the correct way of doing it, but at least it works, and I hope it saves you from any headaches :D.

Making Suwako Moriya’s Hat: Part 1

Suwako Moriya and her hat

Suwako Moriya and her hat

Some months ago, around October, I was Skyping with a friend late at night and we started to talk about Touhou, Suwako Moriya and Arduinos. Having wanting to do something with Arduinos for a long while—my last project being my RFID-enabled door lock—I decided make myself Suwako’s hat, but with LED’s inside the eyes of the hat. My plan is to wear it at NärCon, Sweden’s largest convention.

The next day I went to a local store that sold Arduinos and a small variety of different components, and bought myself an Arduino Uno and a small 8×8 LED matrix to test with. The staff at the store didn’t know what color it was since it wasn’t specified anywhere, but it turns out it was a red LED matrix, not exactly what I wanted since I wanted RGB, but it was good enough to test some ideas with.

Next up was finding something to make the eyes from, where I could mount the LED matrices inside. It was hard to find, but in the end I found a Gashapon-style vending machine at my local food store that had large enough capsules to fit LED matrices inside. I bought two of them and planned to paint them white.

The LED matrix had some issues though, when put close to a surface, the dots would bleed together, not creating the effect I want.

I decided to continue with just creating something with LED matrices and worry about the actual hat and eyes later on. Since I didn’t like the LED matrix being single color, I decided to take a look at RGB LEDs. That’s when I found NeoPixels, and while the same store I bought the original matrix had some NeoPixel products, they didn’t have the 8×8 RGB NeoPixel Matrix. Fortunately a Swedish online store does stock them and other Adafruit products, so I ordered two 8×8 NeoMatrix and I had them two days later. Hooking them up to the Arduino wasn’t all that hard but I had to replace all my code so far to use the NeoPixel and NeoMatrix library instead.

These things are bright!

These things are bright!

One issue though, according to the specifications of the LED matrices they use a lot of power, about 60mA per LED that’s set to full brightness on white, that’s 3.84A for one panel and 7.68A for two panels, if all LEDs were set to full white! Fortunately I wont be using all LEDs at once, but even 32 LEDs on full light (16 LEDs on each panel) would end up at 2A. An AA battery for example would discharge in about 1 hour at that rate, but I don’t expect to use full brightness (the above picture is at 3% brightness), so it’ll probably last for a while. Since I’ll be wearing this hat for 4 days at NärCon, having it powered on when I’m not sleeping, I got myself two 12000mAh USB battery packs just in case. The plan was to use both batteries at the same time, one powering one of the LED matrices and the Arduino itself, and one powering the other matrix. So I ended up buying two mini-USB breakout boards to tap into the power of the batteries, unfortunately I never got it to work since I don’t think those battery packs are made for stuff like this, only one of the LED matrices would power on (occasionally, both matrices would power on), and with my limited knowledge of electronics I couldn’t figure it out, perhaps it was an issue with how I connected the grounds. In the end I decided to use only one battery pack for everything while the other is charging.

I started making a simple animation with regular eyes that randomly looks around. The plan is to have different kind of animations and different kind of eyes, as well as different colors.

But having it work on its own is boring, right? Remote control! I bought a cheap IR receiver and control set at the local store, planning to use that to control which eyes are on and what colors, but that only gives limited options and would be awkward to use. So I head back to Lawicel, the Swedish retailer of Adafruit stuff, and bought the nRF8001 Bluetooth LE chip! The plan is to use my OnePlus One to control eyes, colors and other stuff, and when that’s implemented I would head on to make an Android Wear app for my LG G Watch R so I can control the hat from my watch. Implementing Bluetooth was quite hard at a first glance, especially since I don’t really know C++. I found some examples that I tried to modify to fit my project, but it just wouldn’t work, the LED matrices would just flash and misbehave. Took me a week of thinking to figure out what was wrong, and in the end it turns out I had ran out of RAM, since the Arduino only has 2KB of RAM and I was storing the eye patterns in a bad way, using around 192B RAM, which I later could optimize down to only 24B after someone in #arduino pointed out that I should use bitshifting instead.

Android app draft

Android app draft

The Android application is currently pretty much a draft and not much to see. When it’s done you should be able to select which eye to draw on each matrix or to enter random mode. You should also be able to select the color, or put it in fade/random color mode.

The hard part for me would probably be how to make the actual hat and eyes. For the eyes I’ve been thinking 3D printing would be my only choice, and I have to find a makerspace that would let me use their 3D printer for this. I have done some 3D modeling in Autodesk Maya trough my high school years, but it was quite a while ago and this time I’d have to use something free like Blender. The fact that I never done anything to be 3D printed before makes it even harder. My idea so far is to make a sphere that can be taken into two parts, much like those Gashapon capsules. I’m uncertain on how to solve the color bleed issue, but the plan was to have small pipes from each LED on the matrix to the surface of the sphere, but I’m unsure if that would work.

I have released all the source code on GitHub, but be warned I don’t know much about C++ and electronics like these, so there might be some weird stuff going on.

Lastly, below is a video of my progress so far with “random mode” and color fading. It still needs a lot of tweaks, obviously the eyes should not change that often. Currently it has a 3% chance of changing every tick, and there are about 50  – 60 ticks every second, so it changes pretty fast, but it’ll be updated later on to a much saner value. Stay tuned for part 2!

One-click copy paste

Previously I’ve created an AutoIt script to enable one-click copy paste but unfortunately I lost it in my old blog. By request I’ve created a new, and in my opinion even better than the previous one.

Source

To use it, simply select some text, press the hotkey and press the hotkey again to start pasting. By default, the hotkey is F4. Below is the source (I leave it up to you to install AutoIt).

Save this as an .au3 file and you should be able to run it by double-clicking on it (a good idea is to drop it in the auto-start folder if you don’t want to start it manually each time you start). It is possible to compile the .au3 file to a standalone executable that doesn’t require AutoIt to be installed, I haven’t done so as anti-virus and anti-spyware generally blocks it and you’d have to jump trough hoops to enable it.

Download

Here is a direct download for the .au3 file if you don’t want to copy/paste the source code above.

Download “One-click copy paste” OneClickCopyPaste.au3 – Downloaded 1170 times – 1 KB

What I’ve learned during GSoC and phpBB

During this summer I’ve been working on an REST API for phpBB as part of my Google Summer of Code project. I’ve learned a lot about a lot of things during the summer and I’ve surely had one of the best summers in a while! I wanna give out a thanks to the phpBB team, and summarize what I’ve learned and done during the summer.

What I’ve learned

I’ve learned a lot about the open source world which I hadn’t really entered before. Mainly in this regard I’ve learned more about Git and how to do proper pull requests and such (also a lot about rebasing, branching and other useful things I’ve never had done before). I’ve gotten more used to read others code and understand their code as well as modifying it. I’ve also got to pickup the phpBB coding guidelines and it has helped a lot to improve my code.

Another thing I’ve learned (but still need to delve deeper into) is Symfony, composer and dependency injection. Prior to GSoC I had barely used any of them. I managed to use what I learned in other projects almost instantly as I got a new job as a part time web developer at a local company, and we use a lot of these things there (but using Laravel instead of Symfony).

I’ve learned a bit about cryptography and hashing, to use for authenticating the API. To be clear I’m still confused with all of this and probably have to take another look at it later, but I definitely know more about it now than I did before.

I’ve learned a lot about various design practices, and new stuff like repositories, entities and so on, which was new to me. This too has helped me at my new job as we use these kinds of things a lot.

Thanks

I want to thank the phpBB team for accepting my proposal, it has been one of the greatest summer in a long time for me. I’ve learned a lot of useful stuff and I’ve met a lot of new people in the process. I would also like to say thanks for the generous invite to the hackathon in Montreal, it was really fun and I would do it again if I ever get the chance in the future.

A last word

It seems like I’ve given up on the project but I sure haven’t, just that I’ve been busy with some courses at college as well as that I got a new part time web development job taking away my time. I’m still thinking about the API and still planning the next steps, and I promise to finish what I’ve started. I’m currently planning and preparing for my final thesis work this spring, but I probably have some free time now and then when I can devote myself to the API again, don’t worry :).

GSoC, phpBB API and hackathon in Canada!

The results from GSoC (Google Summer of Code) has been out for almost 2 weeks now but I haven’t had time to write about it yet. I was one of the six that get to work for phpBB and I’m really excited to do this! As I’ve mentioned earlier I tried to apply to phpBB with the API last year but didn’t have any luck back then but this year I was lucky to be selected.

What I’m gonna do this summer is to create a REST API for phpBB. It would allow developers to create various applications that can interact with phpBB in an easy way. Examples could be mobile applications, desktop applications and more. If you wanna check out my proposal, it’s publicly available here. I think it’s gonna be a lot of fun creating this and I hope it turns out awesome.

Furthermore, I’ve been invited to phpBBs hackathon in Montreal, Canada in July! The event is between July 10 – 14, but I’ve decided to stay a bit longer until July 19 to sightsee a bit and meet up with a friend of mine. It’s gonna be a lot of fun, I’ve only been on a smaller hackathon before that I hosted myself. It will be my first trip outside Europe, I think it’s gonna be awesome.

Open source – Start contributing early

I’ve come to a realization that I suck at contributing to open source projects. It’s not like I dislike contributing to open source or believe I suck at coding, but that I just don’t know what to contribute, how to do it and feeling insecure about my code and solutions. I became interested in programming in 2007 and since then I’ve only coded alone on my own projects. I never contributed to open source projects because the thought never crossed my mind.

It wasn’t until last year when my friend Callum Macrae showed me GSoC (Google Summer of Code) that I (slowly) started to contribute to open source projects. The goal of GSoC is to submit a proposal of how you would make any of the registered open source projects better, and if you are chosen by the open source organization, you’ll implement your proposal during the summer and you get paid by Google for doing so. Last year I submitted a proposal to phpBB to create an API, but I didn’t get chosen (looking back at that proposal I realize how bad that proposal was so it wasn’t a surprise I wasn’t chosen).

To increase ones chance to get selected you’d have to contribute to the project beforehand. I checked the bug tracker for things I could fix but come to a realization: I have no idea what I am doing. As I’ve only worked alone for so long I always knew what bugs there were in my projects and how to solve them, but this wasn’t the case anymore. I checked bug report after bug report but I wasn’t secure enough to fix any of them. Fortunately I found a couple of rather easy ones to get a hang of the open source workflow of phpBB. Here was my next problem: how exactly do I submit my patch? phpBB uses GitHub, and I’ve used git and GitHub before but only for my own use. I read some instructions on the phpBB wiki and asked Callum as he contributes to phpBB as well, and sure enough, just fork the repo, branch from the correct develop branch, fix whatever bug you were about to fix, commit that to your own repo and submit a pull request. Easy enough.

Or not. No one had ever reviewed my code before. No one had ever commented on it. This had led me to become very insecure about my code and I was worried what others would think about my solutions and my coding style. I was unsure what I would put as my commit message, and sometimes even afraid to send the pull request. I managed to fix some four or so small bugs but after my proposal was rejected I got a bit bored with contributing and forgot about it and went back doing my own stuff instead.

This year I’ve tried again with GSoC and phpBB with same idea but a much better proposal than last year and I’m in the same position and I’m trying to get used contributing again. I still have issues with insecurity and how I should solve things the best way, but I’m trying my best to overcome these obstacles and get more active contributing to various open source projects.

The message I want to send is to start as early as possible with contributing to open source projects. Even if you don’t know enough programming yet, check the bug tracker occasionally, try to understand parts of the projects code and when you find a bug report that seems easy enough for your skill level, don’t be afraid to fix it! I believe by contributing to open source projects, you’ll learn to be a better team member and you get valuable feedback on your work. It’s a crucial skill and you really need it later in life when you are working in teams at whatever job you get.

I know for one that currently I’m not the best at working in teams. I’m doing ok with groups of 2 – 3 persons, but when working in large groups I have a hard time making myself useful for the rest of the team. I do wish I’ve started contributing to open source projects earlier in my life so I would be more prepared at it when starting college, but I guess it’s better late than never am I right?

BitPoint, a simple bitcoin betting game

The front page of BitPoint

The front page of BitPoint

I’ve been working on a small project last week, I call it BitPoint. It’s a pretty straight forward gambling game paid by bitcoins. A short summary of bitcoins for those who do not know: It’s a online, decentralized cryptocurrency that can be used to buy stuff from other bitcoin users. It can be exchanged to other currency by trading with other users (and by the time writing this 1BTC is worth USD20). If you want to know more, check the link above.

How it worksYou start of by buying a point in the field (by clicking anywhere). Each point costs a small amount of BTC (the cost per next point increases for each point, if first point costs 0.005btc, next point will cost 0.005+0.005btc). The bitcoins from each point is added to the jackpot. Each day at midnight UTC a random point is selected on the field with a certain radius. Every point inside this radius is a winner and will split 70% of the jackpot amongst them. 20% of the jackpot goes to the next round and 10% to me (the jackpot number on the front page already takes this into account and that is the actual amount that will be split!). If no one wins, 100% of the jackpot goes to the next round.

So why don’t you go check it out!

Python: Getting the title of windows, getting their processes and their commandlines using ctypes and win32

My latest assignment for Commie was to create a plugin for xchat to list running encoding processes in the staff IRC channel. You can write plugins in a few various languages and me knowing a little bit of Python I chose to do it that way.

The problem I was facing was to find all windows of x264, fetch their titles (which included how far the encoding was, time remaining etc), and output each title to the channel when a user would say “.encoding”.  Easy enough, I thought, I’ll just fetch all processes and check if “x264” is in the title. I found some methods and code snippets using pywin32 (win32gui, included in pywin32). Unfortunately, xchat only supports Python 2.6 (which is a rather old version of Python) and the windows installer for pywin32 for Python 2.6 is broken and I didn’t manage to compile from source either, so I had to find an alternate way of doing it without any libraries.

Luckily, I found a way of getting all windows titles using ctypes (which is included in Python by default) and the win32 API (thanks to this blog post). It worked really well and from there I managed to easily get the titles I wanted and send them to the IRC channel. Unfortunately, the titles didn’t contain what was encoding, so it would be pretty useless to know that something is 50% done.

Looking around a bit, I managed to find a way to get what was being encoded. The commandline of the process that started the encoding included the directory name of what was encoding. Now I needed to somehow fetch the commandline, but I only had a window and not a process. Having the window I obviously had the window handle. By reading some documentation I found the GetWindowThreadProcessId, which returns a process id from a window handle. Now that I have that, how do I easily get the CommandLine? The easiest way I could figure out was to use WMIC, a commandline interface to WMI (a type of query language to fetch various stuff from the OS, something in the lines of that). From this on it was only some formatting left to do. Below is the full code. Hopefully others can use it to understand how it works, as I had some troubles searching for the right way to do this (all thought it may be a rare thing to do).