Founding AI Engineer
fiska
Job Description
- Agentic Systems: Take agentic AI SMMP workflows from concept to production, owning the complete pipeline including retrieval, reasoning, tool integration, and safety measures while continuously improving accuracy through data-driven iteration
- Evaluation Infrastructure: Architect robust evaluation frameworks for prompts and models with both offline and online testing capabilities, enabling rapid experimentation through A/B testing and systematic performance comparison
- Context Engineering: Build sophisticated context management systems incorporating RAG, long-term memory, intelligent summarization, and data compression to enhance both conversations and automated expense categorization
- Product Collaboration: Partner directly with founders and product leaders to transform customer pain points into intelligent agent capabilities, run experiments, and deliver production features
- Internal Tools: Develop reusable tools and SDKs that empower engineering teams and AI systems to build faster, creating a multiplier effect across the organization
- Model Orchestration: Manage integrations across multiple LLM providers with intelligent fallback mechanisms, balancing cost efficiency, response speed, and output quality for optimal performance
- Observability: Implement comprehensive monitoring infrastructure tracking agent behavior, API costs, failure patterns, hallucination detection, and safety guardrails with actionable dashboards driving product strategy
- Safety & Compliance: Design and deploy robust guardrails, and implement safety protocols ensuring AI-generated outputs meet stringent financial industry compliance and regulatory requirements
- Continuous Improvement: Establish clear success metrics covering accuracy, latency, cost, and safety, design and execute A/B tests, and integrate user feedback loops to drive measurable quality improvements