Apna built one of India’s leading professional marketplaces on the cloud, growing to 22 million users in just three years across 70+ cities, with a unique AI job seeking algorithm. Today, it is looking towards global expansion with Vertex AI.
In 2019, tech entrepreneur Nirmit Parikh went undercover working as an electrician in Mumbai, to discover firsthand the challenges blue-collar workers face in making a living. He found a pervasive lack of support networks and matchmaking tools that white-collar workers enjoy in terms of accessing jobs and upskilling.
The experience inspired Parikh to launch Apna, a smartphone app that leverages proprietary algorithms to drive AI-enabled matches between candidate profiles with employers at a hyperlocal level, covering India’s rural areas as well. Apna also hosts more than 70 professional communities in which everyone from carpenters to electricians and delivery people can build their networks, share insights on finding work and upskilling.
Since its launch, Apna has become one of India’s largest job and professional networks, with 22 million users in more than 70 Indian cities. It enables 18 million job interviews and over 35 million peer-to-peer conversations every month. The success has enabled it to become a unicorn, a startup valued at over $1 billion, within two years of launching.
To succeed in its mission to “democratize the Indian job hunt,” Apna needed an agile cloud infrastructure and powerful data processing tools to scale its solution, and crunch job seeker data from across a nation of 1.3 billion people. It also sought an advanced AI platform for nimble creation of machine learning models, to enable continuous improvement of the app amid rapidly evolving socio-economic conditions.
Apna turned to Google Cloud and a combination of Google Kubernetes Engine (GKE), BigQuery and Vertex AI, and unlocked the solutions it needed to open an aspirational future for Indian jobseekers, with targeted matchmaking and networking.
“If you have a smartphone with Internet, Apna will guarantee you a level playing field. Through our AI algorithm, we provide a virtual visiting card that matches skills with relevant jobs,” says Suresh Khemka, Head of Platform Engineering and Infrastructure, Apna. “That’s made possible through cloud native capabilities in Google Cloud, including GKE, BigQuery and Vertex AI.”
Building the future of intelligent job searches on agile cloud infrastructure
Apna’s proprietary matchmaking algorithm enables what Khemka calls “screened job leads” that weed out irrelevant job posts and applicants from a sea of undifferentiated employment data. From the start, Apna felt it needed to be a cloud native app that lives exclusively on Android handsets, India’s dominant smartphone platform due to affordability. It also wanted the scalable agility gained from being powered by Kubernetes-based microservices.
Khemka says tapping Google Cloud as its cloud provider became a natural choice, because both Kubernetes and Android are solutions developed by Google. Building its infrastructure on GKE enabled agile interaction between all Google Cloud managed services, as well as seamless Android compatibility, with the latest features and technology. The clincher, says Khemka, was the flexibility and intuitive use that Google Cloud offered, compared with other cloud vendors.
“We can create a Kubernetes cluster in four minutes with Autopilot, GKE’s new operational mode, which takes our infrastructure agility to another level,” says Khemka. “Then when we’ve built a new app feature, it takes no more than 10 minutes to onboard the service onto GKE and deploy to the public.”
Unleashing the power of ML modeling with a fast, intuitive AI platform
With a lean data science team, Apna knew that building machine learning models from scratch with traditional data analytics engines would be a laborious and time-consuming task. When the startup decided to give Vertex AI a try, it was surprised at how the team was able to simply start building the Apna algorithm with little manual work, getting it up and running within weeks.
“Data scientists traditionally spend the majority of time obtaining the data, massaging the data, removing outliers, performing everything that comes under the category of machine learning (ML) Ops,” says Khemka. “Vertex AI automated most of that burden, making everything simple for us.”
Khemka says one of the most impressive aspects of Vertex AI is how it constantly allows Apna to experiment with new AI features to fine-tune the jobmatching solution on a daily basis. This feature is particularly important because employer and candidate needs are in constant flux, amid the ups-and-downs of the pandemic and global economy.
“With Vertex AI we’re able to build models and deploy them fast,” says Khemka. “It’s so simple to use that we can run multiple ML experiments on a daily basis, testing them live on a small proportion of users, gauging metrics and making rapid tweaks. That’s how we constantly fine-tune our platform.”
The AI modeling that Apna carries out on Vertex AI is turbocharged by CloudSQL as its fully managed database, combined with Pub/Sub, its data integration service, to enable pipelines for millions of daily data points. This datastream can then be fed through BigQuery for powerful analytics at unlimited scale, which in turn allows Vertex AI to perform ML modeling with virtually zero DevOps intensity. With BigQuery, Khemka says, Apna is able to crunch up to 500 million user interactions every day to power its Vertex AI-enabled algorithms.
“Traditionally, you would have to perform data analytics and build ML models by hand, and that’s beyond the capacity of our small team. The same goes with managing infrastructure,” says Khemka. “Google Cloud’s managed services make things easy, freeing us to concentrate on the creative side of designing algorithms.”
“Data scientists traditionally spend the majority of time obtaining the data, massaging the data, removing outliers, performing everything that comes under the category of machine learning (ML) Ops. Vertex AI automated most of that burden, making everything simple for us.”
—Suresh Khemka, Head of Platform Engineering and Infrastructure, Apna
Results-based empowerment for a continuously evolving job platform
The results of running the platform on Google Cloud managed infrastructure services and Vertex AI have been substantial. Khemka estimates that Apna is compressing the time to build AI models by 20%, compared to how long it would take to build from scratch using a traditional data analytics engine.
This enables Apna to launch up to seven AI experiments per day on a limited proportion of users, gauging metrics, analyzing feedback and tweaking the system. “Vertex AI gives our data science and engineering teams creative independence,” says Khemka. “Experimentation becomes a no-brainer for us, thanks to the ease of Vertex AI and the instant ML modeling it enables.”
Meanwhile, Khemka estimates that the team is saving up to 40% of DevOps time through a combination of managed services. These include infrastructure provisioning and autoscaling on GKE, live application monitoring on Cloud Operations (formerly Stackdriver), and crash-reporting and bug fixing with Firebase Crashlytics.
Fostering growth of safe communities with AI-driven content analytics
A key part of the Apna solution is community building. Peer learning offers an invaluable resource for finding jobs and sharing tips on everything from learning English to interview techniques, to optimize the chance of success in job hunting.
However, as with most community empowered forums, it is not free from detractors who abuse the platform, be it with fake job postings or bullying comments. Apna leverages Vertex AI to create ML models that intelligently detect abusive or fraudulent behavior based on keywords, with a high degree of accuracy.
Thanks to the possibility of rapid ML modeling on Vertex AI, in a situation where fraudsters constantly evolve methodologies, Apna is able to identify and remove up to 60% of inappropriate content on the platform daily, estimates Khemka.
“Vertex AI allows us to ensure the trust and security of our platform at scale, as we continue to grow at a rapid rate,” says Khemka. “This is an ability we keep evolving every day with new ML models so we can stay ahead of the fraudsters.”
“Traditionally, you would have to perform data analytics and build ML models from scratch, and that’s beyond the capacity of our small team. The same goes with managing infrastructure. Google Cloud’s managed services make things easy, freeing us to concentrate on the creative side of designing algorithms.”
—Suresh Khemka, Head of Platform Engineering and Infrastructure, Apna
Supporting global expansion with built-in security and network flexibility
Apna now has ambitions to grow beyond India to reach a billion job seekers worldwide. As it pursues these plans, it feels confident it will be able to harmonize the different data and privacy regulations of new markets, thanks to the secure-by-design data protection protocols embedded in the Google Cloud ecosystem.
In particular, Google Cloud’s deployment of subnets in virtual private networks (VPNs) promises to enable a high degree of security, by controlling traffic to-and-from instances with network firewall rules in different cloud regions. Anthos will also support Apna’s global expansion, Khemka says, by enabling compatibility with multi cloud, hybrid cloud, or on-premise environments whenever necessary.
“Google Cloud brings us more than powerful, agile and scalable managed services,” concludes Khemka. “It gives us the network security and flexibility we need to succeed in any region’s cloud or regulatory environment as we plan our global expansion.”
