Serving a Model as an API: Turn a Prediction into a Contract
Another program needs your prediction
A flat-listing page has a size entered by a user. The rent pipeline lives in Python. An API gives the page a defined way to send the size and receive an estimate without knowing how the estimator was trained.
API means Application Programming Interface. In this lesson, it is an HTTP service: a client sends a request to a path, and the service returns a response. The prediction code becomes one step inside that request handling.
1. Receive JSON
{ "area_sqft": 750 }
The named field declares square feet, not acres or square metres.
2. Validate
Required field, numeric type, finite value, permitted range
Invalid requests return a client error before calling the model.
3. Predict
Loaded pipeline.predict([[750]])
The serving process loads once; it does not fit on each request.
4. Respond
rent_inr: 25,500; model_version: rent-v1
The known toy output includes its target unit and release identity.
Validation failure → 422 in the example. A successful 200 response still needs outcome evaluation.
Start with a request and response
Our contract accepts area_sqft as a finite number measured in square feet. A request such as {"area_sqft": 750} becomes one feature row: [[750]]. The response includes rent_inr and model_version so the amount and producing release are explicit.
1import json
2
3request_body = '{"area_sqft": 750}'
4request = json.loads(request_body)
5feature_row = [[request["area_sqft"]]]
6response = {"rent_inr": 25500.0, "model_version": "rent-v1"}
7
8print("Feature row:", feature_row)
9print("Response JSON:", json.dumps(response, sort_keys=True))Feature row: [[750]]
Response JSON: {"model_version": "rent-v1", "rent_inr": 25500.0}This first cell only demonstrates JSON, the text format used to exchange named values. The response amount is chosen from our known toy prediction. The next example actually loads and runs a fitted model.
Load at startup, not for every request
Reading the model file for every request adds unnecessary work. Fit offline, save the artifact, then load it when the serving process starts. If loading fails, do not announce that the instance is ready for predictions.
Key Points
- Startup establishes which artifact the instance serves.
- Request handling validates and predicts, but does not fit again.
- Each worker process usually has its own loaded copy and memory cost.
- A new release can start new instances rather than mutating a loaded model during a request.
Build and exercise a real FastAPI endpoint
FastAPI is a Python web framework. Pydantic describes and validates request fields. TestClient sends HTTP-style requests inside the Python process, so this cell can run in Colab without opening a port. Its small training fixture creates the artifact only to make the example standalone.
1import subprocess
2import sys
3try:
4 from fastapi import FastAPI
5 from fastapi.testclient import TestClient
6 from pydantic import BaseModel, ConfigDict, Field
7except ImportError:
8 subprocess.check_call(
9 [sys.executable, "-m", "pip", "install", "-q", "fastapi>=0.115", "pydantic>=2", "httpx>=0.27"],
10 stdout=subprocess.DEVNULL,
11 )
12 from fastapi import FastAPI
13 from fastapi.testclient import TestClient
14 from pydantic import BaseModel, ConfigDict, Field
15
16from contextlib import asynccontextmanager
17from pathlib import Path
18from tempfile import TemporaryDirectory
19import joblib
20from sklearn.pipeline import Pipeline
21from sklearn.preprocessing import StandardScaler
22from sklearn.linear_model import LinearRegression
23
24class RentRequest(BaseModel):
25 model_config = ConfigDict(extra="forbid")
26 area_sqft: float = Field(strict=True, ge=100, le=5000, allow_inf_nan=False)
27
28class RentResponse(BaseModel):
29 rent_inr: float
30 model_version: str
31
32with TemporaryDirectory() as directory:
33 artifact = Path(directory) / "rent-v1.joblib"
34 pipeline = Pipeline([("scale", StandardScaler()), ("model", LinearRegression())])
35 pipeline.fit([[500], [700], [900], [1100]], [18000, 24000, 30000, 36000])
36 joblib.dump({"pipeline": pipeline, "model_version": "rent-v1"}, artifact)
37
38 @asynccontextmanager
39 async def lifespan(app):
40 app.state.bundle = joblib.load(artifact)
41 yield
42
43 app = FastAPI(lifespan=lifespan)
44
45 @app.get("/ready")
46 def ready():
47 return {"ready": True, "model_version": app.state.bundle["model_version"]}
48
49 @app.post("/predict", response_model=RentResponse)
50 def predict(request: RentRequest):
51 bundle = app.state.bundle
52 rent = float(bundle["pipeline"].predict([[request.area_sqft]])[0])
53 return {"rent_inr": round(rent, 2), "model_version": bundle["model_version"]}
54
55 with TestClient(app) as client:
56 print("Ready status:", client.get("/ready").status_code)
57 good = client.post("/predict", json={"area_sqft": 750})
58 print("Prediction status:", good.status_code)
59 print("Prediction:", good.json())
60 print("Missing field:", client.post("/predict", json={}).status_code)
61 print("Wrong type:", client.post("/predict", json={"area_sqft": "750"}).status_code)
62 print("Out of range:", client.post("/predict", json={"area_sqft": -1}).status_code)
63 print("Extra field:", client.post("/predict", json={"area_sqft": 750, "acre": 1}).status_code)Ready status: 200
Prediction status: 200
Prediction: {'rent_inr': 25500.0, 'model_version': 'rent-v1'}
Missing field: 422
Wrong type: 422
Out of range: 422
Extra field: 422A status of 200 means this request succeeded. Here 422 means the body failed the declared input schema. strict=True prevents silently turning numeric strings into numbers; extra="forbid" rejects unexpected fields. The 100-to-5,000 range is a teaching contract, not a range learned from the model.
Valid syntax is not proof of a supported prediction
A size of 4,000 is syntactically allowed by this API but far outside our toy training range of 500 to 1,100. Production needs a policy for inputs outside the evaluated range: reject, warn, fall back, or route for review.
Validation should also reject missing values, non-finite numbers, impossible units, and inconsistent fields. Document which values are accepted rather than hoping the model will infer the intended meaning.
1import math
2
3def valid_area(value):
4 if isinstance(value, bool) or not isinstance(value, (int, float)):
5 return False
6 return math.isfinite(value) and 100 <= value <= 5000
7
8for name, value in [("ordinary", 750), ("missing", None), ("NaN", float("nan")), ("boolean", True)]:
9 print(f"{name}: accepted={valid_area(value)}")ordinary: accepted=True missing: accepted=False NaN: accepted=False boolean: accepted=False
This standalone validator demonstrates the rule without starting a service. NaN means “not a number”, a special floating-point value. Ordinary JSON does not support NaN or infinity, so an HTTP client should not send them as JSON numbers.
A notebook test is not a deployed server
TestClient verifies the application contract without a network server. To serve real clients, put the application in a module, provide a persistent artifact path, and run an application server such as Uvicorn. For a module named app.py exposing app, a local command is uvicorn app:app --host 0.0.0.0 --port 8000.
Do not run that long-lived server command as an ordinary Colab teaching cell. In the next lesson, it becomes the start command of a packaged service. The model fits in the training job, not when the server receives traffic.
Return enough context to investigate a result
Key Points
- A request ID connects one response with its operational record.
- The response unit identifies what the number means.
- The model version identifies the fitted artifact.
- A feature schema version identifies the input interpretation.
- Error responses should help clients correct invalid requests without exposing internal tracebacks.
Record request timing, version, status, and suitable feature summaries for monitoring. Decide what input data genuinely needs to be retained rather than logging every user field by default.
Requests have operational limits
Prediction latency includes parsing, validation, transformations, estimation, and serialisation. Large batches or many workers can increase memory and CPU use. Set limits for request size and batch length, measure under load, and decide what to return when the instance cannot serve more work.
For external clients, the deployment also needs controlled access and encrypted transport. An endpoint working in a test process has not established those properties. Start with the smallest service that meets the intended prediction workflow.
Where people get confused
Key Points
- The request handler predicts; it should not fit a new estimator for each request.
- A 200 response means the request completed, not that the estimate is accurate.
- A schema-valid number can still be outside the range where the model was evaluated.
- TestClient exercises the app in-process; it does not demonstrate a public deployment.
Quick check
What should a prediction API contract define?
It should define feature names, units, types, missing and invalid-value behaviour, allowed request shape, response units, and model or schema version information.
Why load the artifact at startup?
Loading once avoids repeated file work and allows the instance to become ready only after it has the intended model available.
What does a 422 response mean in this FastAPI example?
The request did not satisfy the declared input schema, so the prediction function was not called with that invalid input.
What does a successful TestClient call prove?
It provides evidence that the application handles that request as expected in-process. It does not establish network deployment, load capacity, or model quality.
An interface you can explain
Key Points
- Define a clear input and output contract.
- Load the trusted fitted pipeline once per serving process.
- Validate before prediction and identify the serving release.
- Test valid, invalid, and unsupported cases explicitly.
- Next, package and release the service so another machine can run it.
Theia
Can you explain this concept?
Practise Serving a Model as an API: Turn a Prediction into a Contract with realistic interview questions. Choose spoken or typed answers, get feedback on your reasoning, then return to the material that needs work.
Sign in to practise this conceptThe session stays focused on this lesson. Your selected interview level sets the bar.