Core ML

REST API Serving

A model is just a math equation. To actually let users interact with it, you must wrap it in a standard web server so that mobile apps and websites can send it data over the internet and receive predictions back.

The user's mobile app sends a JSON payload over the internet via HTTP. The REST API receives it, extracts the data, passes it to the ML model for a prediction, and returns the result as a JSON response.

Why Does This Exist?

You have trained a brilliant computer vision model that detects diseases in plant leaves. You serialized it to ONNX, and you wrote a Python script that loads the model and successfully analyzes a local image file.

But how does a farmer standing in a field in Iowa use it? They can't run your Python script on their iPhone. The industry standard way to connect software applications across the internet is via a REST API (Representational State Transfer Application Programming Interface).

By wrapping your model inside a REST API, you create a dedicated URL (an "endpoint") on your server. Any mobile app, website, or other software can send data to this URL and receive an instant prediction back, regardless of what programming language they are written in.

Think of It Like This

Imagine you are an incredibly smart chef who makes the best pizza in the world (the ML Model). However, you only speak Italian, and you are locked in the kitchen.

To serve customers, you need a waiter (the REST API). The waiter takes the customer's order in English, walks into the kitchen, translates it into Italian for you, waits for you to cook the pizza, and then delivers it back to the customer. The waiter is the standardized interface that bridges the outside world to the kitchen.

How It Works

A typical ML REST API is built using lightweight web frameworks like FastAPI (Python), Flask (Python), or Express (Node.js).

When a user wants a prediction, their device sends an HTTP POST Request to your server. The request contains a payload of data formatted in JSON (JavaScript Object Notation), which is a universal text format that all programming languages understand.

The REST API performs three jobs:

Validation: It checks the incoming JSON. Did the user actually send an image, or did they accidentally send a PDF? If it's invalid, it immediately rejects it with a 400 Bad Request error.
Execution: It hands the validated data to the ML model and waits for the inference to finish.
Response: It takes the mathematical output of the model (e.g., [0.1, 0.9]), formats it into a human-readable JSON response ({"disease": "blight", "confidence": 0.9}), and sends it back over the internet with a 200 OK success code.

Show Me the Code

Here is how you wrap a pre-trained PyTorch model into a production-ready REST API using FastAPI.

1from fastapi import FastAPI, HTTPException2from pydantic import BaseModel3import torch4
5# 1. Initialize the Web Server6app = FastAPI(title="Plant Disease API")7
8# 2. Load the model once into memory when the server starts9model = load_my_pytorch_model()10model.eval()11
12# 3. Define the exact JSON shape we expect from the user13class PredictionRequest(BaseModel):14    pixel_data: list[float]15    crop_type: str16
17# 4. Create the Endpoint URL (/predict)18@app.post("/predict")19def get_prediction(request: PredictionRequest):20    try:21        # Extract data from the JSON request22        input_tensor = torch.tensor(request.pixel_data)23        24        # Run the ML model (The actual math)25        with torch.no_grad():26            output = model(input_tensor)27            28        # Format the result back into JSON29        return {30            "disease_detected": True if output > 0.5 else False,31            "confidence": float(output)32        }33        34    except Exception as e:35        # If anything breaks, tell the user gracefully36        raise HTTPException(status_code=500, detail="Inference Failed")

Watch Out For

The Global Interpreter Lock (GIL) Bottleneck. In Python, the GIL prevents multiple threads from executing Python code at the exact same time. If your web server receives 50 requests simultaneously, and your model takes 1 second to run, the 50th user will have to wait 50 seconds to get their response! Python REST APIs (like FastAPI) are great for routing requests, but they are terrible at concurrent heavy math. For high-traffic production, you must use a dedicated Inference Engine (like Triton or vLLM) behind your API, or rely on asynchronous dynamic-batching.

The Quick Version

A REST API acts as a bridge, allowing users across the internet to query your ML model.
Users send data (like text or images) via an HTTP POST request formatted as JSON.
The API validates the input, runs the model, and returns a JSON response.
FastAPI is the modern standard for writing ML APIs in Python because it automatically validates the incoming JSON data using Pydantic.
Wrapping a model in a Python API is easy, but scaling it to handle thousands of simultaneous users requires advanced serving architectures to overcome Python's concurrency limitations.