BrickPi Candy Sorter

Site: GoLabs
Course: BrickPi Candy Sorter
Book: BrickPi Candy Sorter
Printed by: Guest user
Date: Sunday, 21 July 2024, 3:21 AM


Sort your Halloween candy with Google Cloud Vision and Raspberry Pi.

1. Introduction

In this tutorial, we’re going to build a machine to sort your Halloween candy, using the Raspberry PiBrickPi, some LEGOs, and Google Cloud Vision.

Pile of candy too big this year?

Need to sort the junk from the gold? Have a certain candy you hate? Fear not, the BrickPi Candy Sorter is here to help! In this tutorial we’ll harness the powerful Google Cloud Vision, BrickPi, Raspberry Pi, Raspberry Pi Camera, and a pile of LEGO bricks to build a machine to find what you want (and don’t want) automatically!

After you run the BrickPi Candy Sorter, you’ll have a pile of candy you want, and the junk you can give your little brother.

“Robots are coming for Dirty, Dangerous, and Dull Jobs”; sorting candy isn’t dangerous or dirty (I hope), but it is kind of dull. The BrickPi Candy Sorter using Google Cloud Vision solves this problem!

2. Tools and Materials

You will also need a Google Cloud account. Fear not, we have instructions here on how to set up a Google Cloud Account, and the first 60 days are free! You can use your existing Google or Gmail account to set it up, and it takes about 10 minutes to set up.

3. A Quick Overview

First, we’ll make a Python list of the candy we like and a list of the candy we don’t like. After we fire up the Python program, you’re going to dump your unsorted candy into the machine. The individual candy bars will be sorted onto a conveyor belt. The conveyor belt dumps the piece of candy into the imaging basket.



Once your candy is dumped into the imaging basket, the Ultrasonic Sensor is tripped and the BrickPi turns off the conveyor belt. In the imaging basket, the Raspberry Pi camera will take a picture of the candy bar and upload it to the Google Cloud Vision, asking for a brand label.

Google Cloud Vision will search the image and return its best guess for what the brand is of the object (candy) in the picture.

Finally we’ll compare the value returned by Google Cloud Vision to a list of candy we want to keep, and a list we want to give to our brother. The candy sorter will dump in one direction if it’s a keeper, and another if it’s a reject! As a final touch, the speaker will yell out loud whether it was Good Candy or Bad Candy, and what brand candy it found.

4. Build the Sorting Machine

Build the Google Cloud Vision and the Raspberry Pi Camera Sorting Machine.

5. Wire Up the Raspberry Pi

To finish the project, you’ll need to connect a few things:

  • Conveyor Belt Motor - Motor Port B.
  • Imaging Basket Servo Motor Port C.
  • Rake (the motor on top) - Motor Port D.
  • Raspberry Pi Power Supply - Micro USB Port of the Raspberry Pi.
  • BrickPi Power Pack - Power Port of the BrickPi.
Remember to connect the Camera! We’ll be using a Raspberry Pi Camera with the Raspberry Pi, here are some directions on setting up the camera.

These Raspberry Pi Camera set up directions assume you’re using Raspbian for Robots on your SD Card. You can buy an SD Card with Raspbian for Robots on it, or you can download it for free and install it using our directions here.

 If you’re not using Raspbian for Robots, you will need to take additional steps to enable and set up your camera; here are some directions on setting up the camera.

6. Prepare the Raspberry Pi

Since we’re using Raspbian for Robots, there are only a few changes that need to be made to the software. All of them are specific to using Google Cloud Vision on the Raspberry Pi. We have a very thorough tutorial on how to get set up here. In short you’ll need to run the following in the command line:

sudo pip install --upgrade pip
sudo apt-get install libjpeg8-dev
sudo pip install --upgrade google-api-python-client
sudo pip install --upgrade Pillow
sudo apt-get install python-picamera, espeak

Finally, you’ll need to upload your Google Application Credentials in JSON to the Raspberry Pi. Use an FTP program (such as FileZilla) or for Windows users Samba (

) to move the JSON file over to your Raspberry Pi. Place the JSON file in the home directory (“/home/pi”).

Next, we’ll turn on Super User and make our credentials available. In the command line, type the command “su”:


You may need to run:

sudo su

and you may need to enter in your password.

Next, we’ll make the JSON credentials available to Python. In the command line, run the following:


Be sure to substitute your JSON filename in this command with the name of the file you have on your Raspberry Pi.

Finally, we’ll download the code. Again, this should go into the home directory:


7. List your Gold (or Trash)

Opening up the Python program:


sudo nano

On lines 30 and 31 display two lists: one of good candy (“good_candy”) and one of bad candy going to our little brother Steve (“bad_candy”).


good_candy = ['Mars','Milky Way','Bounty']


bad_candy = ['Juicy Fruit','Doublemint', 'Snickers']

Add all the candy names you like. Note that the code will default to “bad” so if it’s not on the list, it’s not going into the good pile! In my example, I am not really interested in gum, and I hate Snickers. On the other hand, I love Bounty, Mars, and Miky Way, which are all in the Good Candy pile!

8. Run the Program

Make sure you’re still in Super User mode. If you see the word ‘root’ at the beginning of the prompt, then you’re already in Super User mode. If you don’t see ‘root’, run the command line





And the response should look something like this:

9. More Candy?

Aside from the cute tricks with LEGO, some awesome computing power is going on. In this project, we’ve harnessed some high end computing power to do a simple task: sort your candy. We’re using a lot of computing power: a Raspberry Pi linked to the Cloud and accessing Google Vision – that’s a whole lot of computers for sorting candy! Humans are still in charge of the robots and candy-sorting is serious business when you’re two years old!

While the task may seem simple, that’s kind of the point. This tutorial demonstrates a really simple but boring task that can be automated with Google Cloud Vision. Making this technology accessible is important, and combining Google Cloud Vision and the Raspberry Pi Camera does that!

The robots are coming, and they’ll probably be running Google Cloud Vision!

10. More Questions?

11. Bonus

Watch a quick video of my 2 year old son explaining the machine here.