Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
What are augmented Images?
3.
Limitations for Augmented Images
4.
What is ARCore?
5.
Requirements for a good image
5.1.
Creating java class for Augmented Image
5.2.
Setting up Database
5.3.
Creating 3d Model
6.
Frequently Asked Questions
7.
Conclusion
Last Updated: Mar 27, 2024
Easy

Augmented Images

Master Python: Predicting weather forecasts
Speaker
Ashwin Goyal
Product Manager @

Introduction

Adding 3D images to your surroundings is called Augmented Reality. The technology is now even available on mobile phones. For instance, we all use filters when taking pictures; these pictures can add a cap, animated ears, and big eyes to our faces when they detect our surroundings; this is how augmented reality works. Using this technique, augmented images can be created.

What are augmented Images?

Augmented Image allows you to develop AR apps that respond to specific 2D images, such as posters. Pointing the camera at particular pictures can kick off AR experiences - for example, pointing the camera on the road, and a character or animal will appear in 3D on the screen. The app can then use augmented reality to place a 3D model over a simple 2D image turned into an augmented image.


This uses reference images and ARcore tracking technology to find the location of those images in an AR session. You need to follow the steps for creating an augmented reality app.

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Limitations for Augmented Images

You will be able to determine if augmented images are appropriate for your app if you understand the limitations of augmented images.

  • A maximum of 1000 reference images can be stored in augmented images.
  • Arcore can't track more than 20 images simultaneously.
  • It is also impossible for Arcore to track multiple instances of the same Image. 
  • An ideal physical environment would be 15 x 15 centimeters and flat.
  • Images that are moving cannot be tracked by ARcore but can be tracked again when they stop moving.

What is ARCore?

ARcore is a company that offers an augmented images-a database that contains reference images that, when viewed by the device's camera, produce augmented images when in the user's environment. This allows you to create anchor points on top of the images that are being tracked by the AR code.

Google has developed ARCore, while Apple has developed ARkit. Both companies have been heavily investing in Augmented Reality. Individual developers can now harness the power of AR through these technologies, which was unthinkable only a few years ago.

To integrate virtual content with the real world, ARCore uses three leading technologies:

Motion Tracking: It helps the phone determine its position in the world.

Environmental understanding: It provides your phone with the ability to identify all types of surfaces and their sizes, whether vertical, horizontal, or angled.

Light Estimation: The phone can estimate the environment's current lighting conditions by using this feature.

A database of images can be created offline, or images can be added in real-time as they are captured. A pose will be assigned to each of these images once ARCore detects the images in an area and the boundaries around the Image.

Requirements for a good image

  • The camera frame must be filled with at least 25% for initial detection.
  • Keep the camera in a clear view. It should not be seen if the camera is moving too fast or partially obscured, or viewing from a highly oblique angle.
  • A minimum of 300 x 300 pixels is required for the size of the Image.
  • PNG or JPEG images can be provided as reference images.
  • It does not utilize color information. Grayscale and color equivalents can be used as reference images or on the fly by users.
  • Images that are compressed heavily may interfere with feature extraction.
  • Detection and tracking will be compromised if an image has many geometric features or few features.
  • Repeated patterns also pose a problem when trying to detect and track images.
  • To determine the quality of each Image, use the ARCore SDK's Arcoreimg tool. Scores above 75 are recommended.

Creating java class for Augmented Image

In order to modify some properties of the default fragment, we need a custom fragment. To begin, create a class named "AugmentedImage" to extend the ArFragment and implement the getSession method.

package com.example.augmentedimages;
import com.google.ar.core.Config;
import com.google.ar.core.Session;
import com.google.ar.sceneform.ux.ArFragment;
public class AugmentedImage extends ArFragment {
    @Override
    protected Config getSessionConfiguration(Session session) {
        return super.getSessionConfiguration(session);
        
        //create an config object
        Config config=new Config(session);
        config.setUpdateMode(Config.UpdateMode.LATEST_CAMERA_IMAGE);
        config.setFocusMode(Config.FocusMode.AUTO);
        session.config(config);
        this.getArtSceneView().setupSession(session);
    }
}


Inside MainActivity.java

package com.example.augmentedimages;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.net.Uri;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import com.google.ar.core.Anchor;
import com.google.ar.core.AugmentedImageDatabase;
import com.google.ar.core.Config;
import com.google.ar.core.Frame;
import com.google.ar.core.Session;
import com.google.ar.core.TrackingState;
import com.google.ar.sceneform.AnchorNode;
import com.google.ar.sceneform.FrameTime;
import com.google.ar.sceneform.Scene;
import com.google.ar.sceneform.rendering.ModelRenderable;
import java.util.Collection;

//to check if our image bitmap is being tracked
public class MainActivity extends AppCompatActivity implements Scene.OnUpdateListener{
    private AugmentedImage ai;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        ai = (AugmentedImage) getSupportFragmentManager().findFragmentById(R.id.fragment);
        //setting the scene to OnUpdate
        ai.getArSceneView().getScene().addOnUpdateListener(this);
    }
    public void setupDatabase(Config config, Session session){
        //create a bitmap for the image
        Bitmap imap = BitmapFactory.decodeResource(getResources(), R.drawable.cat);
        //creating augmented image database
        AugmentedImageDatabase aid = new AugmentedImageDatabase(session);
        //adding bitmap of the image to the database using addImage() method
        aid.addImage("cat",imap);
        //adding database to Configuration
        Config.setAugmentedImageDatabase(aid);
    }
   }

Setting up Database

Now call the setupDatabase method into AugmentedImage.java.

// calling the method from MainActivity.java
((MainActivity) getActivity().setupDatabase(config, session));
//to check if our image bitmap is being tracked
public class MainActivity extends AppCompatActivity implements Scene.OnUpdateListener{
//setting the scene to OnUpdate
ai.getArSceneView().getScene().addOnUpdateListener(this);


In MainActivity.java create the onUpdate method

@Override
    public void onUpdate(FrameTime frameTime) {
        //to create a frame everytime there is a new scene
        Frame frame = ai.getArSceneView().getArFrame();
        //collecting all the tracked images
        Collection<AugmentedImage> i=frame.getUpdatedTrackables(AugmentedImage.class);
        //To go through every image to know if the image is being tracked.
        for(AugmentedImage image: i){
            //to check if the image is tracked
            if(image.getTrackingState()== TrackingState.TRACKING){
                //if this is the image then proceed further
                if(image.getName.equals("pic")){
                    //creating an anchor at the center of the image
                    Anchor anchor = image.createAnchor(image.getCenterPose());
                    //creating a new method for 3D Model
                    createModel(anchor);
                }
            }
        }
    }

Creating 3d Model

Now call the setupDatabase method into AugmentedImage.java.

Then inside MainActivity.java, create a createModel method that constructs a renderable from a Uri provided. The rendered renderable is passed into a method that adds a node to the scene by attaching it to a node and placing it onto the scene.

And after constructing an AnchorNode from an anchor, a renderable is attached to another node, which is added to the AnchorNode, and the AnchorNode is added to the scene.

private void createModel(Anchor anchor) {
    ModelRenderable.builder().setSource(this, Uri.parse("cat.sfb")).build().thenAccept(modelRenderable -> placeModel(modelRenderable, anchor));
}
private void placeModel(ModelRenderable modelRenderable, Anchor anchor) {
    //creating an anchor node
    AnchorNode anchorNode = new AnchorNode(anchor);
    anchorNode.setRenderable(modelRenderable);

    //placing this model to the scene by calling arFragment
    arFragment.getArSceneView().getScene().addChild(anchorNode);
}

 

Run your app to see how it works. ARCore will detect the feature points and add your 3D Model to the reference image when it detects the actual Image. The app uses ARCore by Google and Sceneform SDK to create our first Augmented Images app.

Also See, Image Sampling 

Frequently Asked Questions

  1. What are AR images?
    By adding digital content to a live camera feed, augmented reality appears as though the virtual content is part of the physical world around you. 1. For example, this might include transforming your face into a giraffe or overlaying digital directions on real-world streets.
     
  2. When does AR work on a phone?
    Digital content is merged with the natural world through augmented reality. There are no headsets, goggles, or other extra gear required with this game compared to virtual reality (VR). The only thing you need is an AR app and the camera on your device.
     
  3. What is the process of tracking AR images?
    2D images can be detected, tracked, and augmented using Image Tracking. Multiple targets, also known as multiple targets, provide the ability to track multiple images simultaneously. It can recognize up to 1000 images offline, and there are thousands of target images hosted in the cloud.
     
  4. Is AR an app-dependent technology?
    It has become impossible due to ground-breaking AR technology. Augmented reality experiences do not require an app to be created or enjoyed. Embraced by marketers and consumers alike, web-based augmented reality is the hottest new thing on the block.

Conclusion

In this blog, we have seen how Augmented Images can be created using the ARcore, introduced by Google.

We hope that this blog has helped you enhance your knowledge about Augmented Images and if you would like to learn more, check out our articles on the link. And if you have a passion for AR and VR, visit this article on Snake Snack Game

You can use the Coding Ninjas Studio Guided Path to learn data structures and algorithms, programming languages, etc. Test your coding skills by participating in the Coding Ninjas Studio contests and tests! You can look at these challenges, interview experiences, and interview bundles for placement preparations if you are just starting out in your learning process. 

Do upvote our blog to help other ninjas grow. Happy Coding!

Previous article
Anchors and Trackables
Next article
Occlusion
Live masterclass