Endexia Local AI Edge Device

Explore Endexia, a local AI edge device integrating NVIDIA and Raspberry Pi. Learn about its custom RAG, sandboxing, and use as a decentralized hub for open and closed-source models.

Gemma 3 NVIDIA Jetson Ollama RAG REST API

Overview

Endexia is a local Ai device that helps streamline workflows, works with open and closed source models, and can be used a decentralized hub station. We plan to show how we’ve designed and integrated the devices hardware using NVIDIA and Rasberry Pi, integrated a customized RAG and sandboxing.

Links

https://endexia.tech
Endexia is an on-device Jetson-powered hardware system for service dispatch.

Tech stack

Gemma 3

Google DeepMind's most advanced open-weight model: multimodal, multilingual, and optimized for single-accelerator deployment.

Gemma 3 is Google DeepMind's latest family of open-weight models, built on the same technology powering the Gemini 2.0 series. It delivers state-of-the-art performance for its size, available in four variants (1B, 4B, 12B, and the flagship 27B) to ensure efficient deployment across diverse hardware, from phones to single-GPU workstations. Key technical upgrades include a massive 128K-token context window and integrated multimodal capabilities, allowing for analysis of text, images, and video. The model also boasts unparalleled multilingual support, covering over 140 languages, making it a powerful, portable, and responsibly developed foundation for global AI applications.

https://deepmind.google/technologies/gemma/

View projects
NVIDIA Jetson

NVIDIA Jetson is the leading platform for edge AI and robotics: compact, power-efficient modules accelerating autonomous machine development.

Jetson is a high-performance, small-form-factor computing platform for AI at the edge, delivering GPU-accelerated processing for robotics and embedded systems. The platform includes a range of modules—from the accessible Jetson Orin Nano up to the powerful Jetson AGX Thor, which delivers up to 2070 FP4 TFLOPS for complex generative AI workloads. Every module is a complete system (CPU, GPU, memory, interfaces) supported by the unified NVIDIA JetPack SDK: this includes a Linux OS, CUDA-X accelerated libraries, and APIs for deep learning and computer vision. Developers leverage this integrated hardware and software stack to rapidly prototype and deploy advanced AI solutions in manufacturing, healthcare, and logistics.

https://developer.nvidia.com/embedded/jetson-modules

View projects
Ollama

Deploy and run open-source Large Language Models (LLMs) like Llama 3 and Mistral locally on your machine: achieve private, cost-effective AI via a simple command-line interface.

Ollama is the essential tool for running LLMs locally: consider it the Docker for AI models. It packages complex models and dependencies into a single, easy-to-use application for macOS, Linux, and Windows systems. You get immediate access to models like Gemma 2 and DeepSeek-R1 via a straightforward CLI or REST API. This local-first approach guarantees data privacy and security, eliminating cloud dependency and high API costs. Ollama also optimizes performance on consumer hardware using techniques like quantization, ensuring efficient execution even on standard desktops.

https://ollama.com

View projects
RAG

RAG (Retrieval-Augmented Generation) is the GenAI framework that grounds LLMs (like GPT-4) on external, verified data, drastically reducing model hallucinations and providing verifiable sources.

RAG is a critical GenAI architecture: it solves the LLM 'hallucination' problem by inserting a retrieval step before generation. A user query is vectorized, then used to query an external knowledge base (e.g., a Pinecone vector database) for relevant document chunks (typically 512-token segments). These retrieved facts augment the original prompt, providing the LLM (e.g., Gemini or Llama 3) the specific, current, or proprietary context required. This process ensures the final response is accurate and grounded in domain-specific data, avoiding the high cost and latency of full model retraining.

https://en.wikipedia.org/wiki/Retrieval-augmented_generation

View projects
REST API

REST API (Representational State Transfer) is an architectural style defining a uniform, stateless interface for web services, leveraging standard HTTP methods (GET, POST, PUT, DELETE) to manage resource-based data.

REST is an architectural style, not a protocol, mandating six constraints for building scalable, reliable distributed systems (client-server, stateless, cacheable, layered system, uniform interface, and code-on-demand: optional). Its core strength is the uniform interface, which maps CRUD operations (Create, Read, Update, Delete) directly to standard HTTP verbs. Specifically, POST creates a new resource (e.g., to `/users`), GET retrieves a resource (e.g., from `/users/101`), PUT/PATCH updates it, and DELETE removes it. This resource-centric, stateless approach ensures predictable communication and high interoperability across diverse clients and servers.

https://developer.mozilla.org/en-US/docs/Glossary/REST

View projects

Endexia Local AI Edge Device

Related talks