An algorithm for mobile vision-based localization of skewed nutrition labels that maximizes specificity - Emerging Trends in Image Processing, Computer Vision, and Pattern Recognition - page 226

2 Previous work
In 2006, our laboratory began to work on ShopTalk, a wearable system for independent blind
supermarket shopping [ 6 ]. In 2008-2009, ShopTalk was ported to the Nokia E70 smartphone
connected to a Bluetooth barcode pencil scanner [ 7 ] . In 2010, we began our work on computer
vision techniques for eyes-free barcode scanning [ 8 ] . In 2013, we published several algorithms
for localizing skewed barcodes as well as horizontally or vertically aligned NLs [ 5 , 9 ] . The al-
gorithm presented in this chapter improves the previous NL localization algorithm by relax-
ing the NL alignment constraint for up to 35-40° in either direction from the vertical orienta-
tion axis of the captured frame.
Modern nutrition management system designers and developers assume that users under-
stand how to collect nutritional data and can be triggered into data collection with digital
prompts (e.g., email or SMS). Such systems often under-perform, because many users find it
difficult to integrate nutrition data collection into their daily activities due to lack of time, mo-
tivation, or training. Eventually, they turn of or ignore digital stimuli [ 10 ] .
To overcome these challenges, in 2012 we began to develop a Persuasive NUTrition Man-
agement System (PNUTS) [ 5 ] . PNUTS seeks to shift current research and clinical practices in
nutrition management toward persuasion, automated nutritional information extraction and
processing, and context-sensitive nutrition decision support. PNUTS is based on a nutrition
management approach inspired by the Fogg Behavior Model [ 10 ], which states that motiva-
tion alone is insufficient to stimulate target behaviors. Even a motivated user must have both
the ability to execute a behavior and a trigger to engage in that behavior at an appropriate
place or time.
Another frequent assumption, which is not always accurate, is that consumers and patients
are either more skilled than they actually are or that they can be quickly trained to obtain the
required skills. Since training is difficult and time consuming, a more promising path is to
make target behaviors easier and more intuitive to execute for the average smartphone user.
Vision-based extraction of nutritional information from NLs on product packages is a funda-
mental step in making proactive nutrition management easier and more intuitive, because it
improves the user's ability to engage into the target behavior of collecting and processing nu-
tritional data.
3 Skewed NL localization
3.1 Detection of Edges, Lines, and Corners
Our NL detection algorithm uses three image processing methods: edge detection, line detec-
tion, and corner detection. Edge detection transforms images into bitmaps where every pixel
is classified as belonging or not belonging to an edge. The algorithm uses the Canny edge de-
tector (CED) [ 11 ] . After the edges are detected (see Figure 2 ) , the image is processed with the
hough transform (HT) [ 12 ] to detect lines (see Figure 3 ) . The HT algorithm finds paths in im-
ages that follow generalized polynomials in the polar coordinate space.