Digital Color Book

User research

Our project began with a comprehensive literature review of relevant research and studies on color learning for young children. To further enhance our understanding, we conducted interviews with a preschool teacher and visited the Tekniska museum in Malmö, where we were inspired by the art exhibition (figures 1 and 2). Through this research, we gained valuable insights, including:

• Active learning and direct experience of objects, people, ideas, and events are beneficial for cognitive restructuring.

• Children explore the world through direct manipulation.

• Stimulating children’s interests is done through challenges and feedback.

• Active learning improves children’s aspirations, motivation, socialization, and self-esteem.

Ideation: A digital color book

In our ideation session, we brainstormed and visualized a variety of concepts by sketching them on paper. Our goal was to select the concept that resonated with all of us and further develop it. Sketching proved to be an effective method of communicating our ideas, as it allowed us to easily showcase and explain our concepts through visual representations.

I was particularly interested in exploring how children could actively engage with their learning by interacting with their surroundings. I leveraged my creativity and came up with the concept of a digital color book that utilizes computer vision to capture real-life colors. In my initial sketch (figure 3), the computer would focus on specific areas of the drawing and encourage children to fill them in with colors that closely resembled those of real-life objects. Children would hold the objects in front of the laptop camera, and the computer would extract the color of the object and fill the focused area. Once the area was accurately colored, the computer would move on to the next area.

After presenting this concept to my group, we all agreed that the idea of extracting colors from the environment was intriguing and decided to further develop it. (figure 4) We removed the restrictions of realistic colors and the linear focus of the computer, allowing children to be more creative and create unique drawings. We also added a box feature with an internal camera that would help children extract colors. The box had one opening where the children could place objects of the same color. The opening was designed to guide children in understanding where the objects should be placed.

Ideation: The modes.

Our ideation session was filled with creative energy as we brainstormed and sketched out a variety of concepts for our digital color book. We wanted to create a tool that would allow children to actively explore and engage with their environment through the use of vivid colors. One concept that particularly stood out was the idea of using computer vision to capture real-life colors and apply them to a digital coloring book. We envisioned children holding objects in front of a laptop camera, as the color of the object would be extracted and used to fill in specific parts of a drawing.

To add an element of fun and challenge, we decided to include two different modes in our color book: Free coloring mode and memory mode (figure 5). In free coloring mode (figure 6), children could let their imaginations run wild and color the drawings however they pleased. In memory mode (figure 7), children would be presented with a drawing where each part contained randomized preset colors. They would have just 10 seconds to study and memorize the colors before the drawing would lose its colors. Then, they would use their memory to recreate the drawing and evaluate their attempt against the original. This added an extra layer of engagement and excitement to the coloring experience.

Figure 6. The user interface of the coloring mode showcases the free coloring mode and one of the drawings. The dynamic colored circle displays for the user which color is currently the main color.

Figure 7. A GIF showing the memory mode and its stages.

Prototype development

As one of the programmers on the project, I was heavily involved in bringing the digital color book to life. Utilizing HTML, CSS, and JavaScript, I integrated a JavaScript library called tracking.js to track colors in real-time, using an algorithm that measures the color area with pixels. To ensure accurate color recognition, we increased the number of pixels used in the algorithm, prioritizing the largest color area as the main color. We added a total of 6 colors - green, blue, magenta, yellow, red, and orange - however, we encountered challenges when trying to distinguish between similar colors such as yellow and orange. Despite this obstacle, we persevered and created two drawings for the prototype, one for free coloring mode and another for memory mode. Both drawings were created using Illustrator and converted to SVG files for easy implementation in the code. Each element of the drawings could be filled with colors, providing an interactive and engaging experience for young children.

First user testing

The initial user testing of our prototype (figure 8), involved a 7-year-old girl. At the time, the box for capturing colors had not yet been created, so we had to rely on a white surface to focus the external camera. The child initially struggled with precision when coloring the smaller elements of the drawing. However, she demonstrated patience and determination in her efforts to fill these elements with the right colors. As previously noted, the camera's recognition of colors was not always accurate, leading to occasional display of the wrong color. The girl emphasized the importance of using the correct colors to accurately replicate real life, such as using green for grass and blue for bubbles. To achieve this, she searched her environment for objects with the desired colors and even brought her own box of colorful pens as her toys did not have the required hues. When attempting the memory mode, she found it challenging to remember the colors from the drawing, and even tried restarting the mode only to find that the colors were randomized.

Iteration of the prototype

We made changes to our prototype based on the valuable feedback we received (figure 9). To help with the difficulty in remembering colors, we added a sneak peek button to the memory mode. This allows children to reveal the correct missing colors in the drawing, making it easier to use their memory. We also fine-tuned the color recognition algorithms to make them more reliable. Additionally, we added a physical box with additional lights to the prototype. These lights improve the brightness in the box, making it easier to capture accurate colors. The box was also designed to be more playful and colorful, to provide a sense of "magic" for the children.

Second user testing

The second user test was conducted with two girls, ages 5 and 8 (figure 10), and they were fascinated by the concept of using real-life objects to color digital drawings. During the memory mode, they worked together and discussed which colors should be used for each element of the drawing. They only used the sneak peek button when they were struggling to remember the colors. In the free coloring mode, the girls chose realistic colors just like the girl in the previous user test, for example, they collected a wooden doll to achieve a beige-colored wood pattern for a tree. However, the camera's recognition of shades and patterns was limited and they had to settle for preset colors. Both girls were determined to color every single element of the drawing and collaborated by assigning tasks such as choosing colors, collecting objects, and coloring the elements. They even helped each other position the cursor correctly. Although the color recognition problem was improved, it still persisted and the girls had to bring in a lot of yellow objects to get the correct color.

Learning takeaways

Through my experience, I've acquired a deep understanding of teamwork and iterative design processes. I've honed my skills in creating high-fidelity digital prototypes by utilizing programming languages and a JavaScript library. I have also come to appreciate the value of group brainstorming as it is essential in generating ideas and concepts. Additionally, I have learned the importance of conducting user testing with the correct target audience, as it has taught me that a design can only be validated by its intended users. User feedback plays a crucial role in the process of improving the concept through iterative design.