A machine learning consulting challenge helped e-card platform JibJab overcome technical obstacles and launch a product line.
The Los Angeles-based firm had historically provided prospects a easy interface and cropping software to add photographs, seize head pictures in an oval form and place them in customized digital content material. That technique labored effectively sufficient, however JibJab was launching a brand new providing — a bodily coffee-table guide that referred to as for higher-fidelity pictures.
Matt Cielecki, vice chairman of engineering at JibJab, started exploring machine learning (ML) as the way in which to spice up image high quality. He tinkered with pre-trained fashions however bumped into limitations. Models constructed as tutorial initiatives could not be used commercially, for instance.
Matt Cielecki
Serendipitously, a consultant from Mission, a managed cloud providers supplier additionally in Los Angeles, contacted Cielecki to let him find out about its AI providers. Mission had helped the e-card firm years earlier on an AWS migration and price optimization challenge, he famous. That historical past and the well timed outreach paved the way in which for a brand new initiative: JibJab employed Mission in 2021 to construct a ML algorithm from scratch for image cropping.
Mission constructed the primary model of the algorithm, and the ensuing ML mannequin, in about 9 weeks. But the algorithm has gone by means of a number of iterations since then. The cycle of algorithm testing and mannequin refinement continues.
“We are rolling it out to some customers to get suggestions, and we’re beginning to see some good outcomes,” Cielecki mentioned. “We suppose we now have some room for enchancment, however we’re undoubtedly providing a significantly better product.”
Training and refinement
Mission’s process was to coach a ML pc imaginative and prescient algorithm that would detect faces inside uploaded photographs, precisely crop an individual’s full face and hair and ignore any background parts. The preliminary objective was to realize 85% accuracy with the ML-based image reducing method.
In step one towards that goal, Mission used two annotation instruments — LabelMe and Amazon SageMaker Ground Truth — to label pictures and create an information set for coaching the algorithm. Mission used knowledge augmentation methods, akin to including blur, sharpening and rotation, to develop the labeled knowledge set from 1,000 pictures to 17,000.
Next, Mission used Facebook AI Research’s Detectron2, operating in SageMaker, to detect objects inside pictures and carry out occasion segmentation. The latter went past detection, utilizing a extra granular strategy that helped carefully outline the form of a specific object — in JibJab’s case, an individual’s face and hair.
Ramping up the accuracy of this course of concerned coaching and retraining a ML mannequin. Once Mission put a mannequin by means of its paces on JibJab’s coaching knowledge set, the corporate ran it on new image knowledge. This course of uncovered edge circumstances — conditions wherein the algorithm fell in need of totally figuring out faces and hair. Mission tweaked the coaching knowledge set, which is used to revise the mannequin.
This facet of the initiative drove residence the pivotal function of knowledge engineering in synthetic intelligence. “It modified my perspective on what an AI challenge really is,” Cielecki mentioned. “It’s much less of a coding and extra of an information drawback.”
Ryan Ries
Ryan Ries, who leads Mission’s knowledge, analytics and machine learning observe, cited the significance of a various coaching knowledge set. The coaching course of revealed unanticipated points, akin to shiny mild which washes out a part of an individual’s face or lengthy, flowing hair that defies correct cropping.
Determining why a mannequin failed on sure knowledge units was the trail to enchancment. Ries described the investigative course of as learning, “Why is that this an edge case and why is the algorithm doing what it’s doing and the way do I retrain the information set?”
Current use and future plans
JibJab presently makes use of Mission’s image-cutting technique in its Starring You Books product line. A buyer who desires to create a personalised guide uploads an image, which is saved in AWS S3 bucket. AWS’ pc imaginative and prescient platform, Rekognition, detects a face within the image and conducts an image-quality examine, contemplating, amongst different issues, whether or not the facial image is giant sufficient to realize a superb outcome.
We have further coaching and optimizations to implement earlier than the mannequin is prepared for everybody on a broader scale.
Matt Cielecki Vice president of engineering, JibJab
If the image passes muster, a Detectron2 mannequin performs occasion segmentation to zero in on the face and hair for the ultimate image. The final steps are image post-processing and positioning the image within the customized guide.
Mission’s image-cutting algorithm has reached 90% accuracy. With ongoing enhancements, Mission and its buyer have upped the goal to 95%. As the mannequin improves, JibJab will take into account how the image reducing technique might match into different product traces.
“Our major objective is to get the mannequin in a spot that may generate print-quality head cuts for all of our customers, capturing all the distinctive contours and shapes of their faces,” Cielecki mentioned. “We have further coaching and optimizations to implement earlier than the mannequin is prepared for everybody on a broader scale.”
https://www.techtarget.com/searchitchannel/feature/Missions-machine-learning-consulting-gig-boosts-image
