ImageSleuth uses thousands of “experts” to harvest demographic and other info from images. These experts begin their journey at ImageSleuth University, where they are trained on tens of thousands of human faces. After the training process is complete, the experts are required to pass an exam before they are allowed to participate in the production system. Those that pass are put to work in support of the ImageSleuth API, where the experts get to vote for the feature(s) that they were trained to detect. The system uses these vote counts to determine the degree to which it believes the outcomes. For example, if almost all the experts vote for a “male” outcome, then the belief that the subject is male would be very strong, on the other hand, if almost half the exerts disagreed, the belief would be very low.
ImageSleuth uploads and distributes thousands of faces and hour to our HYDRA system. The ImageSleuth system, which runs in the cloud, is comprised of dozens of HYDRA instances. Each HYDRA instance does the work equivalent of hundreds of human annotators, processing 3 faces per second per core.
First, faces are detected in the photos. Then, each face is subjected to HYDRA analysis, which evokes thousands of “expert” AI processes to determine features like age, race, gender and others. The outcomes of the analysis are combined to create not only a determination of the result, but an assessment of how confident the system is in the outcome. The larger the number of AI experts that agree on an outcome, the higher the confidence HYDRA has in that outcome.
Once the analysis is complete, the results are aggregated into JSON objects and redirected back to the customer.
This may all seem quite complex, but a single HYDRA instance performs tens of thousands of these evaluations every day. We can easily increase throughput by adding more instances in the cloud.
The ImageSleuth service outputs our results using industry standard JSON format, which is standard in many Big Data platforms. This information includes:
The location and size of each face in your image: image coordinates are measured from the upper left hand corner of the image. The width indicates how many pixels to the right of the left side of the image the point is. The height indicates how many pixels from the top of the image the point is. Location measure the distance from the upper left hand corner of the image to the upper left hand corner of the bounding box around the face. The size of the face is the size of the bounding box, where width is measured from left to right and height is measured from top to bottom.
For each face, HYDRA harvests a set of features, e.g. age, race, and gender. Each of the features has an array of values it can be assigned; for example, gender can be “MALE” or “FEMALE”. Each feature outcome also has a belief score that roughly represents the percentage of AI experts that voted for that outcome. Beliefs above .5 represent positive evidence, while beliefs under .5 are negative evidence. In our demo we assign the “winner” to be the outcome with the largest belief, but that may not be the customer’s preference. There may be instances where the largest belief is under .5. Some customers may prefer to label these as “don’t know”. Other customers may consider looking at the top two outcomes (in cases that provide multiple outcomes), or factor in the intensity of belief before adopting any outcome.
ImageSleuth is an API-based web service that extracts demographic and other information from people’s faces. Our easy to use, massively scalable solution provides our customers with a cost effective way of harvesting valuable information from large collections of images and videos. We provide accurate, rapid insights that can be used in retail, financial, entertainment, security, search and other applications.