Skip to content
Krishang ZinzuwadiaStart a conversation

DevSoc Hack · 2nd place

Atlas

Autonomous Task Learning & Action System

A privacy-first desktop agent that turns natural-language requests into verified on-screen actions through a local perception and planning loop.

DevSoc Hack
2nd
desktop execution
Local
01

Context

The challenge

Turning a request into a verified desktop action.

Desktop automation has to connect an imprecise human request to exact interface actions. Atlas was built to keep that loop local while giving the agent enough visual context to inspect the screen, decide what to do, perform the action, and check the result.

Krishang's role

  • Architected the local agent and its perception-to-action loop.
  • Built the Rust-based runtime for isolated task execution and cross-process coordination.
  • Connected the planning stack to PyAutoGUI for desktop control and Tauri for the native interface.
02

System

Architecture

From input to outcome.

  1. 01

    Perceive

    The vision pipeline reads the current desktop state with LLaVA and PaddleOCR.

  2. 02

    Understand

    The system combines the request with the visible interface state before choosing a route forward.

  3. 03

    Plan

    Mistral participates in the planning layer, while the browser agent can use MCP for browser-specific work.

  4. 04

    Act

    PyAutoGUI carries out desktop actions through the isolated runtime.

  5. 05

    Verify

    The loop observes the interface again so execution is followed by a state check.

03

Choices

Engineering decisions

What shaped the build.

01

Keep execution local

Atlas was designed as a privacy-first local agent rather than a remote-control service.

02

Separate interface and agent services

A Tauri v2 and Next.js frontend communicates with a FastAPI backend over WebSockets, while Rust handles the desktop runtime.

03

Use a closed action loop

Perception and verification sit on both sides of planning and action instead of treating a generated plan as the finish line.

04

Result

Recorded outcomes

What the project delivered.

  • Placed second at DevSoc Hack.
  • Delivered a native desktop interface, a browser-focused MCP agent, and a Flutter companion in the public project architecture.
  • Kept desktop execution local while coordinating vision, planning, and action components.

Working stack

  • Python
  • Rust
  • Tauri
  • Next.js
  • FastAPI
  • PyAutoGUI
  • LLaVA
  • Mistral
  • PaddleOCR

Evidence boundary

What this account is based on.

The project record and award record list different months, so this case study deliberately does not assert an Atlas date.