I found some cheap wooden cubes from the dollar store, being on a budget, I decided these would be perfect.
They are far from perfect, they are definitely not true cubes. The fit and finish is very low, to be expected from the dollar store. I'm pretty sure they won't be balanced, some sides are bigger than others. Each side needs to be sanded smooth with 400 grit sandpaper (all I have, I'm sure up to 200 would be fine). Some of them were broken, and some of them I broke. They are very light, probably a soft pine, and they will serve my purpose just fine, for now.
First I decided that I would try carving a stamp out of foam with an xacto knife, and then use acrylic paint to stamp each side of the cubes.
First I tried carving the icons freehand, bad idea. I am not very artistic and they just turned out as blobs. I didn't even take pictures of them, I just went straight to burning them.
I printed out some of the icons, taped it to the foam and tried carving the image. This turned out better:

The result of stamping these, turned out ok, but not great:

I was happy for the most part with the skull and the black shield. I had absolutely no luck with the white shield. The image was just too intricate, I didn't even get one finished to try stamping. I knew I needed a different solution.
I just happen to have a cheap black and white laser printer. The toner is basically just plastic. I discovered that if I print the icons in reverse, I can thermal transfer the toner from the paper to the cube.
All I needed to do was print the icons, place the paper on top of the cube and heat up the area. Using a clothes iron would probably be easiest, I don't have one of those so I used a soldering iron with a chisel tip. This basically melts the plastic toner and transfers it onto the wood. You can then remove the paper, if some of it sticks, some warm water helps to loosen it.
The result was really good, even for my first try. I'm sure that with some practice I'll be able to get them to look even better.
I broke the edges and corners to make them feel better to hold, and applied a clear coat to protect the transfer. Here's the result:

I have found blank wooden dice on Amazon, someday I'll order some and try this process on a better base material.