AI Data Solutions Engineer

 Evaluate generative AI agents, copilots, and LLM-based applications for accuracy, relevance, consistency, and business value.
• Develop AI evaluation methodologies, benchmarks, scorecards, and success metrics.
• Perform qualitative and quantitative assessments of AI-generated responses and recommend improvements.
• Partner with AI developers to improve prompts, retrieval quality, and overall AI performance.
• Assess enterprise Sales and Marketing data for AI readiness, including quality, governance, metadata, and usability.
• Prepare structured and unstructured data to support AI, analytics, and intelligent search.
• Design scalable datasets and data models that improve AI retrieval and business insights.
• Analyze Sales and Marketing data to identify trends and actionable recommendations.
• Develop dashboards, reporting, and analytical models using Python, SQL, Databricks, and Power BI.
• Collaborate with business and technical stakeholders to translate requirements into AI-enabled solutions.
• Document evaluation methodologies, data standards, and best practices.
• Evaluate emerging AI technologies and recommend improvements to enterprise AI capabilities.
Minimum Qualifications
• Bachelor's degree in Computer Science, Data Science, Statistics, Mathematics, Engineering, Information Systems, or related quantitative field with 4+ years of relevant experience, or Master's degree with 2+ years of experience.
• Strong proficiency in Python, SQL, Databricks, and PySpark.
• Experience evaluating AI, machine learning, or advanced analytics solutions.
• Experience preparing enterprise data for AI and analytics use cases.
• Experience with Azure, AWS, or similar cloud platforms.
• Strong analytical, statistical, and problem-solving skills.
• Excellent communication skills and experience working in cross-functional environments.
Preferred Qualifications
• Experience evaluating LLMs and AI agents.
• Experience developing AI evaluation frameworks and performance metrics.
• Experience with Sales, Marketing, CRM, customer engagement, or digital analytics data.
• Experience with Power BI or Tableau.
• Knowledge of prompt engineering, semantic search, vector databases, or knowledge management.
• Experience with Git or other version control systems.
• Exposure to relational and graph database design.
• Familiarity with graph databases such as Neo4j or Amazon Neptune.
• Understanding of graph theory concepts, including centrality metrics and semantic clustering.