The Evolution of Android Camera APIs

The Android camera has evolved over the years to offer multiple cogs and levers to developers to customize the camera viewing and picture-taking experience for their applications.

In modern applications, the camera has become an integral part of interfacing. Whether it is used for video calling, creating content, scanning QR/Bar codes, or playing AR games, having a good camera interface defines the success of the application in many cases. Content creation with filters and various other formats (reels, boomerang) adds quite some value to the application.

Any mobile developer who wants to implement a camera interface in their application needs to understand the intricacies and the techniques used for implementing it. Having good knowledge about the options and the available interfaces within the Android system empowers them to customize this interface and bring the best output for the application.

Support Matrix

Until now the Android SDK has three types of camera APIs.

1. Camera1
2. Camera2
3. CameraX

Camera1 API supports API Level 9-20, Camera2 supports API Level 21 and above, and CameraX supports API Level 21 and above with backward compatibility built-in.

Camera1

The first version of the Android camera API has supported and served its role for all the camera-related functionalities that the hardware had supported for quite some time. Programmatically, the only thing to check is whether the device has a camera feature or not, and use permission in the manifest file.

To either preview or do the camera operations, use SurfaceView, TextureView to render content and instantiate an object of this class and implement the SurfaceTextureListener interface.

Then override the below methods to complete implementation.

/**
Invoked when a TextureView's SurfaceTexture is ready for use
**/
public void onSurfaceTextureAvailable(SurfaceTexture arg0, int arg1, int arg2) {}
/**
* Iinvoked when the specified SurfaceTexture is about to be destroyed.
*/
public boolean onSurfaceTextureDestroyed(SurfaceTexture arg0) {}
/**
* Invoked when the SurfaceTexture's buffers size changed
*/
public void onSurfaceTextureSizeChanged(SurfaceTexture arg0, int arg1,int arg2) {}
/*
* Invoked when the specified SurfaceTexture is updated through updateTexImage().
*/
public void onSurfaceTextureUpdated(SurfaceTexture arg0) {}

As it is deprecated and not in usage, I am not discussing much here in this blog.

Camera2

With evolving camera technology, the Android framework has grown in capability with a wide range of customization options. For example, mobile manufacturers want to assemble multiple cameras for better user experiences and to achieve wide-angle previews, photography features, and multi-camera access at a time.

The Camera2 API is more complex and asynchronous, and has lots of capture controls, states, and metadata. But Camera2 API is more customizable and provides in-depth controls for complex uses like manual exposure (ISO, shutter speed), focus, RAW capture, etc.

Following are steps for using camera2 API

• Start from CameraManager
• Setup the output targets
• Get a CameraDevice
• Create a CaptureRequest from the CameraDevice
• Create a CaptureRequestSession from the CameraDevice
• Submit a CaptureRequest to CaptureRequestSession
• Get the CaptureResults
• Thread maintenance

There are multiple Camera2 API implementations depending on the needs available in so many sources.

Here is an example that captures without preview but runs in the background.

Background Service

/**
 * JobIntentService that captures the picture without preview and runs in the background
 */
class NoPreviewCaptureService : JobIntentService() {
    private val CAMERA = CameraCharacteristics.LENS_FACING_BACK // Can be changed. FACING_FRONT is selfie camera
    private var mImageReader: ImageReader? = null

    override fun onCreate() {
        super.onCreate()
        Log.i(TAG, "onCreate()")
        startBackgroundThread()
    }

    /**
     * Starts a background thread and its [Handler].
     */
    private fun startBackgroundThread() {
        mBackgroundThread = HandlerThread(TAG)
        mBackgroundThread?.start()
        mBackgroundHandler = Handler(mBackgroundThread!!.looper)
    }

    private val cameraStateCallback: CameraDevice.StateCallback =
        object : CameraDevice.StateCallback() {
            override fun onOpened(camera: CameraDevice) {
                Log.i(TAG, "onOpened()")
                mCameraDevice = camera

                try {
                    camera.createCaptureSession(
                        listOf(mImageReader!!.surface),
                        sessionStateCallback,
                        null
                    )
                } catch (e: Exception) {
                    Log.e(TAG, "createCaptureSession Failed" + Log.getStackTraceString(e))
                }
            }

            override fun onDisconnected(camera: CameraDevice) {}
            override fun onError(camera: CameraDevice, error: Int) {}
        }

    private val sessionStateCallback: CameraCaptureSession.StateCallback =
        object : CameraCaptureSession.StateCallback() {
            override fun onConfigured(session: CameraCaptureSession) {
                mCaptureSession = session

                capturePicture()
            }

            override fun onConfigureFailed(session: CameraCaptureSession) {}
        }

    /**
     * This a callback object for the [ImageReader]. "onImageAvailable" will be called when a
     * still image is ready to be saved.
     */
    private val onImageAvailableListener = OnImageAvailableListener { reader ->
        Log.i(TAG, "OnImageAvailableListener")
    }

    /**
     * Return the Camera Id which matches the field CAMERA.
     */
    private fun getCamera(manager: CameraManager): String? {
        try {
            for (cameraId in manager.cameraIdList) {
                val characteristics = manager.getCameraCharacteristics(cameraId!!)
                val cOrientation = characteristics.get(CameraCharacteristics.LENS_FACING)!!
                if (cOrientation == CAMERA) {
                    return cameraId
                }
            }
        } catch (e: CameraAccessException) {
            e.printStackTrace()
        }
        return null
    }

    override fun onHandleWork(intent: Intent) {
        Log.i(TAG, "onHandleWork()")
        Handler(Looper.getMainLooper()).post {
            val manager = getSystemService(Context.CAMERA_SERVICE) as CameraManager
            try {
                if (ActivityCompat.checkSelfPermission(
                        this, Manifest.permission.CAMERA
                    ) != PackageManager.PERMISSION_GRANTED
                ) {
                    Log.i(TAG, "No Camera permissions.")
                }
                this?.let {
                    externalFilesFolder = it.getExternalFilesDir(null)?.absolutePath
                }

                manager.openCamera(getCamera(manager)!!, cameraStateCallback, null)
                val characteristics = manager.getCameraCharacteristics(getCamera(manager)!!)

                val map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)

                // For still image captures, we use the largest available size.
                val largest = Collections.max(
                    Arrays.asList(*map?.getOutputSizes(ImageFormat.JPEG)),
                    CompareSizesByArea()
                )
                mImageReader =
                    ImageReader.newInstance(largest.width, largest.height, ImageFormat.JPEG, 1)
                mImageReader?.setOnImageAvailableListener(
                    onImageAvailableListener,
                    mBackgroundHandler
                )
            } catch (e: CameraAccessException) {
                Log.e(TAG, e.localizedMessage)
            }
        }
    }

    /**
     * Compares two `Size`s based on their areas.
     */
    internal class CompareSizesByArea : Comparator {
        // We cast here to ensure the multiplications won't overflow
        override fun compare(lhs: Size, rhs: Size) =
            signum(lhs.width.toLong() * lhs.height - rhs.width.toLong() * rhs.height)
    }

    /**
     * Fragment methods
     */
    companion object {
        private val TAG = "CaptureService"

        /**
         * Conversion from screen rotation to JPEG orientation.
         */
        private val ORIENTATIONS = SparseIntArray()

        init {
            ORIENTATIONS.append(Surface.ROTATION_0, 90)
            ORIENTATIONS.append(Surface.ROTATION_90, 0)
            ORIENTATIONS.append(Surface.ROTATION_180, 270)
            ORIENTATIONS.append(Surface.ROTATION_270, 180)
        }

        // Reference to the path of external files folder
        var externalFilesFolder: String? = null
        val FILENAME_FORMAT = "yyyy-MM-dd-HH-mm-ss-SSS"

        fun enqueueWork(context: Context, intent: Intent) {
            enqueueWork(context, NoPreviewCaptureService::class.java, 1, intent)
        }

        private fun releaseResources() {
            Log.i(TAG, "Releasing Resources")
            closeCamera()
            stopBackgroundThread()
        }

        private var mCaptureSession: CameraCaptureSession? = null
        private var mBackgroundThread: HandlerThread? = null
        private var mBackgroundHandler: Handler? = null
        private var mCameraDevice: CameraDevice? = null

        /**
         * Stops the background thread and its [Handler].
         */
        private fun stopBackgroundThread() {
            mBackgroundThread?.quitSafely()
            mBackgroundThread = null
            mBackgroundHandler?.removeCallbacksAndMessages(null)
            mBackgroundHandler = null
        }

        /**
         * Closes the current [CameraDevice].
         */
        private fun closeCamera() {
            // stop any existing captures
            try {
                mCaptureSession?.abortCaptures()
                mCaptureSession?.stopRepeating()
            } catch (e: CameraAccessException) {
                Log.i(TAG, "Camera session cannot abort")
            }
            // Close the camera device
            mCameraDevice?.close()
            mCaptureSession?.close()
            mCameraDevice = null
            mCaptureSession = null

            // Close the capture session
        }
    }

    /**
     * Capture request starts here
     */
    fun capturePicture() {
        if (mCameraDevice == null) {
            // Already closed
            Log.i(TAG, "Camera is not available.")
            return
        }

        try {
            val captureRequest =
                mCameraDevice?.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE)
            captureRequest?.let {
                it.set(CaptureRequest.CONTROL_MODE, CaptureRequest.CONTROL_MODE_OFF)
                it.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_OFF)
                it.set(CaptureRequest.SENSOR_SENSITIVITY, 100)
                val exposureTime = 1000 * 1000 * 1000 / 20.toLong()
                it.set(CaptureRequest.SENSOR_EXPOSURE_TIME, exposureTime)

                it.set(CaptureRequest.JPEG_ORIENTATION, 90)
                mImageReader?.let { imageReader ->
                    Log.i(TAG, "Assigning capture stuff")
                    it.addTarget(imageReader.surface)
                    // Need handler here for
                    imageReader.setOnImageAvailableListener(
                        mOnImageAvailableListener,
                        mBackgroundHandler
                    )
                }

                mCaptureSession?.capture(
                    captureRequest.build(),
                    object : CameraCaptureSession.CaptureCallback() {
                        override fun onCaptureCompleted(
                            session: CameraCaptureSession,
                            request: CaptureRequest,
                            result: TotalCaptureResult
                        ) {
                            super.onCaptureCompleted(session, request, result)
                            // We are done
                            Log.i(TAG, "Completed taking all pictures. Move ahead.")
                        }
                    },
                    mBackgroundHandler
                )
            }
        } catch (e: Exception) {
            e.printStackTrace()
            Log.e(TAG, "Capture Session Exception: " + Log.getStackTraceString(e))
        }
    }

    /**
     * This a callback object for the [ImageReader]. "onImageAvailable" will be called when a
     * still image is ready to be saved.
     */
    private val mOnImageAvailableListener = OnImageAvailableListener { reader ->
        Log.i(TAG, "Image is available")

        mBackgroundHandler?.post(ImageHandler(reader?.acquireNextImage()))
    }

    /**
     * Saves a JPEG [Image] into the specified [File].
     */
    class ImageHandler(image: Image?) : Runnable {
        private val mImage = image
        override fun run() {
            mImage?.let { theImage ->
                // Get the byte buffer
                val buffer = theImage.planes[0].buffer
                val bytes = ByteArray(buffer.remaining()).apply { buffer.get(this) }

                // Create time-stamped output file to hold the image
                val photoFile = File(
                    externalFilesFolder,
                    SimpleDateFormat(
                        FILENAME_FORMAT, Locale.US
                    ).format(System.currentTimeMillis()) + ".jpg"
                )

                var output: FileOutputStream? = null
                try {
                    output = FileOutputStream(photoFile).apply {
                        write(bytes)
                    }
                    val msg = "Photo saved : ${photoFile.absolutePath}"
                    Log.d(TAG, msg)
                } catch (e: IOException) {
                    Log.e(TAG, e.toString())
                } finally {
                    mImage?.close()
                    output?.let {
                        try {
                            it.close()
                        } catch (e: IOException) {
                            Log.e(TAG, e.toString())
                        }
                        releaseResources()
                    }
                }
            }
        }
    }
}
}

Calling Service

	val intent = Intent(this, NoPreviewCaptureService::class.java)
	NoPreviewCaptureService.enqueueWork(this, intent)

Declare this service in the manifest.xml

	<service android:name=".NoPreviewCaptureService" android:exported="true" 
    android:permission="android.permission.BIND_JOB_SERVICE" />

CameraX

CameraX is the latest solution for developers who are creating highly specialized functionality for low-level control of the capture flow and when more customization is required. CameraX is built on top of the Camera2 API only.

Advantages:

• Easy to use compared to Camera2
• Backward compatibility as it is part of Android Jetpack to Android 5.0. and is consistent across devices
• Lifecycle aware, no need to maintain open, close the camera or when to create capture sessions

Below is a simple example that captures the picture and camera switch with CameraX.

Declare below dependencies in build.gradle

def camerax_version = "1.0.0-beta12"
// CameraX core library using camera2 implementation
    implementation "androidx.camera:camera-camera2:$camerax_version"
// CameraX Lifecycle Library
    implementation "androidx.camera:camera-lifecycle:$camerax_version"
// CameraX View class
    implementation "androidx.camera:camera-view:1.0.0-alpha19"

Activity Code

	class MainActivity : AppCompatActivity() {
    private var imageCapture: ImageCapture? = null
    private lateinit var outputDirectory: File
    private lateinit var cameraExecutor: ExecutorService
    private val REQUIRED_PERMISSIONS = arrayOf(Manifest.permission.CAMERA)
    private val REQUEST_CODE_PERMISSIONS = 20

    // Setting Back camera as a default
    private var cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA
    private val FILENAME_FORMAT = "yyyy-MM-dd-HH-mm-ss-SSS"
    val TAG = "CameraXCapture"

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        requestPermission()

        // Set up the listener for take photo button
        camera_capture_button.setOnClickListener { takePhoto() }

        switch_btn.setOnClickListener {
            //change the cameraSelector
            cameraSelector = if (cameraSelector == CameraSelector.DEFAULT_BACK_CAMERA) {
                CameraSelector.DEFAULT_FRONT_CAMERA
            } else {
                CameraSelector.DEFAULT_BACK_CAMERA
            }
            // restart the camera
            startCamera()
        }

        outputDirectory = getOutputDirectory()

        cameraExecutor = Executors.newSingleThreadExecutor()
    }

    private fun takePhoto() {
        // Get a stable reference of the modifiable image capture use case
        val imageCapture = imageCapture ?: return

        // Create time-stamped output file to hold the image
        val photoFile = File(
            outputDirectory,
            SimpleDateFormat(
                FILENAME_FORMAT, Locale.US
            ).format(System.currentTimeMillis()) + ".jpg"
        )

        // Create output options object which contains file + metadata
        val outputOptions = ImageCapture.OutputFileOptions.Builder(photoFile).build()

        // Set up image capture listener, which is triggered after photo has
        // been taken
        imageCapture.takePicture(
            outputOptions,
            ContextCompat.getMainExecutor(this),
            object : ImageCapture.OnImageSavedCallback {
                override fun onError(exc: ImageCaptureException) {
                    Log.e(TAG, "Photo capture failed: ${exc.message}", exc)
                }

                override fun onImageSaved(output: ImageCapture.OutputFileResults) {
                    val savedUri = Uri.fromFile(photoFile)
                    val msg = "Photo capture succeeded: $savedUri"
                    Toast.makeText(baseContext, msg, Toast.LENGTH_SHORT).show()
                    Log.d(TAG, msg)
                }
            })
    }

    private fun startCamera() {
        val cameraProviderFuture = ProcessCameraProvider.getInstance(this)

        cameraProviderFuture.addListener({
            // Used to bind the lifecycle of cameras to the lifecycle owner
            val cameraProvider: ProcessCameraProvider = cameraProviderFuture.get()

            // Preview
            val preview = Preview.Builder().build().also {
                it.setSurfaceProvider(viewFinder.surfaceProvider)
            }

            imageCapture = ImageCapture.Builder().build()

            try {
                // Unbind use cases before rebinding
                cameraProvider.unbindAll()

                // Bind use cases to camera
                cameraProvider.bindToLifecycle(
                    this, cameraSelector, preview, imageCapture
                )

            } catch (exc: Exception) {
                Log.e(TAG, "Use case binding failed", exc)
            }

        }, ContextCompat.getMainExecutor(this))
    }

    private fun getOutputDirectory(): File {
        val mediaDir = externalMediaDirs.firstOrNull()?.let {
            File(it, resources.getString(R.string.app_name)).apply { mkdirs() }
        }
        return if (mediaDir != null ∧∧ mediaDir.exists())
            mediaDir else filesDir
    }

    override fun onDestroy() {
        super.onDestroy()
        cameraExecutor.shutdown()
    }

    private fun requestPermission() {
        if (allPermissionsGranted()) {
            startCamera()
        } else {
            ActivityCompat.requestPermissions(this, REQUIRED_PERMISSIONS, REQUEST_CODE_PERMISSIONS)
        }
    }

    private fun allPermissionsGranted() = REQUIRED_PERMISSIONS.all {
        ContextCompat.checkSelfPermission(baseContext, it) == PackageManager.PERMISSION_GRANTED
    }

    override fun onRequestPermissionsResult(
        requestCode: Int,
        permissions: Array out String,
        grantResults: IntArray
    ) {
        super.onRequestPermissionsResult(requestCode, permissions, grantResults)
        if (requestCode == REQUEST_CODE_PERMISSIONS) {
            // If all permissions granted , then start Camera
            if (allPermissionsGranted()) {
                startCamera()
            } else {
                // If permissions are not granted,
                // present a toast to notify the user that
                // the permissions were not granted.
                Toast.makeText(this, "Permissions not granted by the user.", Toast.LENGTH_SHORT)
                    .show()
                finish()
            }
        }
    }
}}

XML Code

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">
    <Button
        android:id="@+id/camera_capture_button"
        android:layout_width="100dp"
        android:layout_height="100dp"
        android:layout_marginBottom="50dp"
        android:elevation="2dp"
        android:scaleType="fitCenter"
        android:text="@string/take_photo"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent" />
    <Button
        android:id="@+id/switch_btn"
        android:layout_width="100dp"
        android:layout_height="100dp"
        android:layout_marginBottom="50dp"
        android:elevation="2dp"
        android:scaleType="fitCenter"
        android:text="Switch"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toStartOf="@id/camera_capture_button"
        app:layout_constraintStart_toStartOf="parent" />
    <androidx.camera.view.PreviewView
        android:id="@+id/viewFinder"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

</androidx.constraintlayout.widget.ConstraintLayout>

Mobile cameras will continue to evolve. Wide cameras, ultra-wide cameras, macro cameras, dual, triple, quad, and Penta back cameras, among other types of cameras, are now often found in mobile phones.