Implementing 3D Model Search Services on AWS Cloud.

  1. Configure AWS Batch, which provides serverless batch computing platform, to run a service that connects to legacy database and download the 3D models files into S3 bucket. You can schedule it to run nightly.
  2. Implement an AWS Lambda function to process the downloaded 3D models in S3 bucket and generate shape data using shape representation algorithms. Generated shape datas should be stored in Amazon DynamoDB. You can configure this Lambda function to trigger for S3 bucket put event.
  3. Implement another AWS Lambda function to create several snapshots of the 3D model at different angles and store them as images in S3 bucket.
  4. Extract features from generated images using a Convolutional Neural Net (CNN) model that is pre-trained on the well-known ImageNet dataset or model you trained and deployed using Amazon SageMaker which is a fully managed machine learning platform that allows to create, train and deploy machine learning models quickly in AWS cloud. Using this model, you can extract image textures, geometric data and metadata and store it in Amazon DynamoDB.
  5. Create another lambda function to enrich shape data generated in step 2 with feature data extracted in step 4. Now shape data is enriched with feature data. Shape data is a set of floating numbers. Next step is group similar shapes together.
  6. Using AWS lambda function, build a reference k-NN index on Amazon OpenSearch Service, which is a fully managed service that makes it easy for you to deploy, secure, and run elastic search cost-effectively at scale. Amazon OpenSearch Service offers k-Nearest Neighbour (k-NN) search, which lets you store shape data as vector and group similar shape data by Euclidean distance or cosine similarity using k-NN algorithm.
  1. Using a web app hosted in S3, you can upload a 3D model object (if you have) or you can draw top, front and side view of the model using a sketcher app and upload the views as image. If you present more view images from different angle, you will get more accurate results.
  2. Uploaded image is sent to AWS Lambda via Amazon API Gateway.
  3. AWS Lambda function will generate shape descriptor for the uploaded model / images and then call Amazon SageMaker real time endpoint to extract feature data.
  4. AWS Lambda function will enrich shape descriptor with feature data.
  5. AWS Lambda function sends query to the k-nearest neighbour in the index in the Amazon Elastic Search Service (Amazon OpenSearch Service). It will return a list of k similar model data and their respective Amazon S3 URIs of the models are returned.
  6. AWS Lambda function generates pre-signed Amazon S3 URLs to return back to the client web application to visualise the similar models.
  1. What is 3D shape descriptor ?
    3D Shape descriptor is a set of numbers to represent the points on 3D model surfaces to capture the geometric essence of a 3D object. It is the compact representations of 3D object and the descriptors form a vector space with a meaningful distance metric.
  2. How to generate 3D shape descriptor ?
    There are many algorithm available to generate 3D shape descriptor. They produce a set of 2D view data which are produced by rotating 3D models at different angles. More views produce more accuracy. Popular algorithms are Light Field Descriptor (LFD) and Multi-View Convolutional Neural Network (MVCNN).
  3. What is pre-trained CNN model?
    A pre-trained model is a model created and trained by someone to solve a problem that is similar to the problem we have. In our case, we can use pre-trained resnet50 convolutional neural network which is trained on more than a million images from the ImageNet database. The resnet50 is available as a built-in algorithm in SageMaker.
  4. What is SageMaker?
    It is a fully managed machine learning service to build and train machine learning models quickly and easily, and then directly deploy them into a production-ready hosted environment.
  5. Amazon Elastic Search Service vs Amazon OpenSearch Service.
    Amazon Elastic Search Service is now called Amazon OpenSearch Service which offers the latest versions of OpenSearch and visualisation capabilities powered by OpenSearch Dashboards and Kibana. It enables you to easily ingest, secure, search, aggregate, view, and analyse large volume of data.
  6. What is k-NN for Amazon OpenSearch Service?
    It allows you search for points in a vector space and find the “k nearest neighbours” for those points by Euclidean distance or cosine similarity.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Nanthan Rasiah

Nanthan Rasiah

AWS APN Ambassador | Solutions Architect | AWS Certified Pro | GCP Certified Pro | Azure Certified Expert | AWS Certified Security & Machine Learning Specialty