Shipping On-Device LLM Chat in React Native
For eBanqo's support product, agents live in the chat screen all day, and the AI features had to feel immediate. A round trip to a server for every message adds latency you can feel, and it fails the moment the network does. So instead of routing everything through an API, I moved intent classification onto the device.
On iOS that meant Core ML, and on Android it meant TensorFlow Lite, both bridged into React Native through native modules I wrote in Swift and Kotlin. The model classifies the intent behind a message locally, so the app can respond, route, or suggest instantly, and the network is reserved for the work that genuinely needs the backend.
The interesting engineering is at the boundary. You want the heavy inference off the JavaScript thread so the UI never stutters, and you want a clean, typed interface between the native model and the React layer so the rest of the app does not need to know which platform it is running on. Getting that seam right is what makes on-device AI feel like a feature rather than a science project.
The result was chat that stayed responsive under load and kept working when connectivity dropped. On-device inference is not the right answer for every model, but for latency-sensitive, privacy-sensitive classification it is often better than the API call people reach for by default.