Discovering Tech and AI

Using the Claude Agent SDK for Non-Coding Workflows

DonvitoCodes — Thu, 30 Oct 2025 12:06:20 GMT

I’ve been exploring the Claude Agent SDK, and I had this idea — why not use it for non-coding workflows instead of relying on other agent frameworks like CrewAI or LangChain?

To validate the idea, I built a simple example: a news researcher agent that finds the latest AI news and translates it into Korean.

The Setup

Here’s the core script:

import asyncio
from claude_agent_sdk import query, ClaudeAgentOptions, AgentDefinition
from claude_agent_sdk.types import McpHttpServerConfig
import os

async def main():
    firecrawl_api_key = os.environ['FIRECRAWL_API_KEY']
    firecrawl_mcp = McpHttpServerConfig(
        type="http",
        url="https://mcp.firecrawl.dev/v2/mcp",
        headers={"Authorization": f"Bearer {firecrawl_api_key}"}
    )

    translator_agent = AgentDefinition(
        description="Translate the content from any language to any other language.",
        prompt="You are an expert language translator.",
        tools=["Read", "Edit", "Bash", "Grep"],
        model="sonnet"
    )

    options = ClaudeAgentOptions(
        model="glm-4.6",
        system_prompt="You are an expert news researcher.",
        permission_mode='bypassPermissions',
        cwd="/Users/melvin/PycharmProjects/ClaudeCodeSDK/output",
        mcp_servers={"firecrawl_mcp": firecrawl_mcp},
        agents={"translator-agent": translator_agent}
    )

    async for message in query(
        prompt=(
            "What are the latest news topics in AI? "
            "Write the results to a markdown file with URLs as references. "
            "Then use the translator-agent to translate the content to Korean "
            "and save it to a separate markdown file."
        ),
        options=options
    ):
        print(message)

asyncio.run(main())

Concept 1: Using MCPs for Data Retrieval

I used the Firecrawl MCP to fetch the latest AI news.

The agent gathered data, summarized it, and wrote the results into a Markdown file — all autonomously.

This shows how an MCP can act like an API plugin layer, enabling agents to perform real-world data collection beyond simple prompts.

Concept 2: Sub-Agents for Specialized Tasks

After gathering the news, I wanted a translated version.

Instead of hardcoding translation logic, I created a sub-agent — the translator-agent — specifically for that purpose.

The main agent then delegated the translation task to the sub-agent.

The output:

ai_news_en.md – English summary
ai_news_ko.md – Korean translation

Why This Matters

The Claude Agent SDK already supports:

Tools (Read, Edit, Bash, etc.)
MCPs (external capability servers)
Skills
Sub-agents

These are the same components other AI agent frameworks build from scratch — but here, it’s all native to Claude’s ecosystem.

With what Claude AI has built, developers and researchers can rapidly compose workflows that go beyond chat — from document generation to automated pipelines.

I used the GLM 4.6 model in this example, but of course, it works perfectly with Claude models like Haiku and Sonnet.

A Great Alternative to AI Agent Frameworks

Frameworks like CrewAI and LangChain are excellent for building complex agent systems — but sometimes, simplicity wins.

The Claude Agent SDK gives you the same building blocks — tools, sub-agents, and external connectors — in a lightweight package that integrates naturally with Claude’s ecosystem.

Building a Java API connecting to LLMs with Spring AI and Ollama local models

DonvitoCodes — Sat, 27 Sep 2025 06:17:54 GMT

Introduction

In the rapidly evolving world of AI, developers often need to integrate multiple AI providers into their applications. Whether you're using local models with Ollama, cloud services like OpenAI, or planning to add Anthropic or Google's Gemini, having a unified interface to manage these providers is crucial.

In this tutorial, we'll build a flexible, extensible AI backend using Spring Boot and Spring AI that can seamlessly switch between different AI providers. We'll implement a clean architecture that makes it easy to add new providers without changing existing code.

What We'll Build

We're going to create a REST API that:

Supports multiple AI providers through a unified interface
Allows dynamic provider and model selection per request
Implements a registry pattern for provider management
Provides proper error handling and validation
Uses Spring AI for simplified AI integration

Here's what our architecture will look like:

graph LR
    Client[Client App] --> API[REST API]
    API --> Registry[Provider Registry]
    Registry --> Ollama[Ollama Provider]
    Registry --> Future[Future Providers]
    Ollama --> LLM[Local LLMs]

Prerequisites

Before we begin, make sure you have:

Java 21 or higher installed
Maven installed
Ollama installed and running (for local AI models)
Your favorite IDE (IntelliJ IDEA, VS Code, etc.)

Step 1: Project Setup

Let's start by creating a new Spring Boot project. You can use Spring Initializr or create it manually.

1.1 Create the Project Structure

mkdir ai-backends-java
cd ai-backends-java

1.2 Create the `pom.xml`


<project xmlns="http://maven.apache.org/POM/4.0.0" 
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
         https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0modelVersion>

    <parent>
        <groupId>org.springframework.bootgroupId>
        <artifactId>spring-boot-starter-parentartifactId>
        <version>3.5.6version>
        <relativePath/>
    parent>

    <groupId>com.aibackendsgroupId>
    <artifactId>aiartifactId>
    <version>0.0.1-SNAPSHOTversion>
    <name>ainame>
    <description>AIBackends Javadescription>

    <properties>
        <java.version>21java.version>
        <spring-ai.version>1.0.2spring-ai.version>
    properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-webartifactId>
        dependency>
        <dependency>
            <groupId>org.springframework.aigroupId>
            <artifactId>spring-ai-starter-model-ollamaartifactId>
        dependency>
        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-testartifactId>
            <scope>testscope>
        dependency>
    dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.aigroupId>
                <artifactId>spring-ai-bomartifactId>
                <version>${spring-ai.version}version>
                <type>pomtype>
                <scope>importscope>
            dependency>
        dependencies>
    dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.bootgroupId>
                <artifactId>spring-boot-maven-pluginartifactId>
            plugin>
        plugins>
    build>
project>

1.3 Create the Main Application Class

Create the directory structure and main class:

mkdir -p src/main/java/com/aibackends/ai

// src/main/java/com/aibackends/ai/AiApplication.java
package com.aibackends.ai;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class AiApplication {
    public static void main(String[] args) {
        SpringApplication.run(AiApplication.class, args);
    }
}

Step 2: Define the Provider Architecture

Now let's create the core architecture that will allow us to support multiple AI providers.

2.1 Create the Provider Interface

First, we'll define an interface that all AI providers must implement:

// src/main/java/com/aibackends/ai/provider/ChatProvider.java
package com.aibackends.ai.provider;

/**
 * Interface for AI chat providers
 */
public interface ChatProvider {

    /**
     * Get a chat response from the AI provider
     * 
     * @param message The user's message
     * @param model The model to use (provider-specific)
     * @return The AI's response
     */
    String getChatResponse(String message, String model);

    /**
     * Get the provider type
     * 
     * @return The provider type enum
     */
    ProviderType getProviderType();

    /**
     * Check if the provider supports a specific model
     * 
     * @param model The model name to check
     * @return true if the model is supported
     */
    boolean supportsModel(String model);

    /**
     * Get the default model for this provider
     * 
     * @return The default model name
     */
    String getDefaultModel();
}

2.2 Create the Provider Type Enum

This enum will represent all supported providers:

// src/main/java/com/aibackends/ai/provider/ProviderType.java
package com.aibackends.ai.provider;

/**
 * Enum representing supported AI providers
 */
public enum ProviderType {
    OLLAMA("ollama"),
    ANTHROPIC("anthropic"),
    GEMINI("gemini");

    private final String value;

    ProviderType(String value) {
        this.value = value;
    }

    public String getValue() {
        return value;
    }

    public static ProviderType fromValue(String value) {
        for (ProviderType type : ProviderType.values()) {
            if (type.value.equalsIgnoreCase(value)) {
                return type;
            }
        }
        throw new IllegalArgumentException("Unknown provider type: " + value);
    }
}

2.3 Create the Provider Registry

The registry will manage all available providers and allow us to retrieve them dynamically:

// src/main/java/com/aibackends/ai/provider/ChatProviderRegistry.java
package com.aibackends.ai.provider;

import org.springframework.stereotype.Component;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;

@Component
public class ChatProviderRegistry {

    private final Map providers = new HashMap<>();

    public ChatProviderRegistry(List chatProviders) {
        // Register all available providers
        for (ChatProvider provider : chatProviders) {
            providers.put(provider.getProviderType(), provider);
        }
    }

    /**
     * Get a chat provider by type
     * 
     * @param providerType The provider type
     * @return The chat provider
     * @throws IllegalArgumentException if provider not found
     */
    public ChatProvider getProvider(ProviderType providerType) {
        return Optional.ofNullable(providers.get(providerType))
                .orElseThrow(() -> new IllegalArgumentException(
                        "Provider not available: " + providerType));
    }

    /**
     * Get a chat provider by string value
     * 
     * @param provider The provider name
     * @return The chat provider
     */
    public ChatProvider getProvider(String provider) {
        ProviderType providerType = ProviderType.fromValue(provider);
        return getProvider(providerType);
    }

    /**
     * Check if a provider is available
     * 
     * @param providerType The provider type
     * @return true if available
     */
    public boolean isProviderAvailable(ProviderType providerType) {
        return providers.containsKey(providerType);
    }

    /**
     * Get all available provider types
     * 
     * @return List of available provider types
     */
    public List getAvailableProviders() {
        return providers.keySet().stream().toList();
    }
}

Step 3: Implement the Ollama Provider

Now let's implement our first AI provider - Ollama, which runs AI models locally.

3.1 Create the Ollama Service

// src/main/java/com/aibackends/ai/service/OllamaChatService.java
package com.aibackends.ai.service;

import com.aibackends.ai.provider.ChatProvider;
import com.aibackends.ai.provider.ProviderType;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.ollama.OllamaChatModel;
import org.springframework.ai.ollama.api.OllamaOptions;
import org.springframework.stereotype.Service;

import java.util.List;

@Service
public class OllamaChatService implements ChatProvider {

    private final OllamaChatModel ollamaChatModel;
    private final List supportedModels = List.of(
            "llama3.2", "llama3.1", "llama3", "llama2",
            "mistral", "mixtral", "codellama", "gemma",
            "phi3", "qwen2.5", "deepseek-coder-v2"
    );

    public OllamaChatService(OllamaChatModel ollamaChatModel) {
        this.ollamaChatModel = ollamaChatModel;
    }

    @Override
    public String getChatResponse(String message, String model) {
        // Set the model in options
        String modelToUse = model != null ? model : getDefaultModel();
        OllamaOptions options = OllamaOptions.builder()
                .model(modelToUse)
                .build();

        // Create a new ChatClient with the specified model
        var chatClient = ChatClient.builder(ollamaChatModel)
                .defaultOptions(options)
                .build();

        return chatClient.prompt()
                .user(message)
                .call()
                .content();
    }

    @Override
    public ProviderType getProviderType() {
        return ProviderType.OLLAMA;
    }

    @Override
    public boolean supportsModel(String model) {
        return supportedModels.stream()
                .anyMatch(m -> m.equalsIgnoreCase(model));
    }

    @Override
    public String getDefaultModel() {
        return "llama3.2";
    }
}

3.2 Create Configuration

// src/main/java/com/aibackends/ai/config/OllamaConfig.java
package com.aibackends.ai.config;

import org.springframework.context.annotation.Configuration;

@Configuration
public class OllamaConfig {
    // Spring AI auto-configuration handles the Ollama beans
    // No manual configuration needed when using spring-ai-starter-model-ollama
}

Step 4: Create the REST API

Now let's create the REST controller that will expose our AI services.

4.1 Create the Controller

// src/main/java/com/aibackends/ai/controller/AIController.java
package com.aibackends.ai.controller;

import com.aibackends.ai.provider.ChatProvider;
import com.aibackends.ai.provider.ChatProviderRegistry;
import com.aibackends.ai.provider.ProviderType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

import java.util.List;

@RestController
@RequestMapping("/api")
public class AIController {

    private final ChatProviderRegistry providerRegistry;

    public AIController(ChatProviderRegistry providerRegistry) {
        this.providerRegistry = providerRegistry;
    }

    @PostMapping("/chat")
    public ResponseEntity chat(@RequestBody ChatRequest request) {
        try {
            // Validate request
            if (request.message() == null || request.message().isBlank()) {
                return ResponseEntity.badRequest()
                        .body(new ErrorResponse("Message cannot be empty"));
            }

            if (request.provider() == null || request.provider().isBlank()) {
                return ResponseEntity.badRequest()
                        .body(new ErrorResponse("Provider must be specified"));
            }

            // Get the provider
            ChatProvider chatProvider = providerRegistry.getProvider(request.provider());

            // Validate model if specified
            if (request.model() != null && !request.model().isBlank() 
                    && !chatProvider.supportsModel(request.model())) {
                return ResponseEntity.badRequest()
                        .body(new ErrorResponse("Model '" + request.model() + 
                                "' is not supported by provider '" + request.provider() + "'"));
            }

            // Get the response
            String response = chatProvider.getChatResponse(
                    request.message(), 
                    request.model()
            );

            return ResponseEntity.ok(new ChatResponse(
                    response,
                    request.provider(),
                    request.model() != null ? request.model() : chatProvider.getDefaultModel()
            ));

        } catch (IllegalArgumentException e) {
            return ResponseEntity.badRequest()
                    .body(new ErrorResponse(e.getMessage()));
        } catch (Exception e) {
            return ResponseEntity.internalServerError()
                    .body(new ErrorResponse("Internal server error: " + e.getMessage()));
        }
    }

    @GetMapping("/providers")
    public ResponseEntity getProviders() {
        List providers = providerRegistry.getAvailableProviders().stream()
                .map(providerType -> {
                    ChatProvider provider = providerRegistry.getProvider(providerType);
                    return new ProviderInfo(
                            providerType.getValue(),
                            provider.getDefaultModel()
                    );
                })
                .toList();

        return ResponseEntity.ok(new ProvidersResponse(providers));
    }

    @GetMapping("/providers/{provider}/models")
    public ResponseEntity getProviderModels(@PathVariable String provider) {
        try {
            ChatProvider chatProvider = providerRegistry.getProvider(provider);

            // For now, return a basic response. In a real implementation,
            // each provider would have a method to list available models
            return ResponseEntity.ok(new ModelsResponse(
                    provider,
                    List.of(chatProvider.getDefaultModel()),
                    chatProvider.getDefaultModel()
            ));

        } catch (IllegalArgumentException e) {
            return ResponseEntity.badRequest()
                    .body(new ErrorResponse(e.getMessage()));
        }
    }

    // Request/Response DTOs
    public record ChatRequest(
            String message,
            String provider,
            String model
    ) {}

    public record ChatResponse(
            String response,
            String provider,
            String model
    ) {}

    public record ErrorResponse(String error) {}

    public record ProvidersResponse(List providers) {}

    public record ProviderInfo(
            String name,
            String defaultModel
    ) {}

    public record ModelsResponse(
            String provider,
            List models,
            String defaultModel
    ) {}
}

Step 5: Configure the Application

Create the application configuration file:

# src/main/resources/application.properties
server.port=8085

spring.application.name=ai-backends

# Ollama configuration
spring.ai.ollama.base-url=http://localhost:11434
spring.ai.ollama.chat.model=llama3.2

# Logging
logging.level.com.aibackends=DEBUG

Step 6: Test the Application

6.1 Start Ollama

First, make sure Ollama is running and has a model installed:

# Install Ollama (if not already installed)
# Visit https://ollama.ai for installation instructions

# Pull a model
ollama pull llama3.2

# Start Ollama (usually starts automatically)
ollama serve

6.2 Run the Application

./mvnw spring-boot:run

6.3 Test the Endpoints

List Available Providers

curl http://localhost:8085/api/providers | jq .

Response:

{
  "providers": [
    {
      "name": "ollama",
      "defaultModel": "llama3.2"
    }
  ]
}

Send a Chat Request

curl -X POST http://localhost:8085/api/chat \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Hello! What is 2 + 2?",
    "provider": "ollama",
    "model": "llama3.2"
  }' | jq .

Response:

{
  "response": "The answer to 2 + 2 is 4.",
  "provider": "ollama",
  "model": "llama3.2"
}

Step 7: Adding New Providers

The beauty of this architecture is how easy it is to add new providers. Let's see how you would add OpenAI support:

7.1 Add the Dependency

Add to your pom.xml:

<dependency>
    <groupId>org.springframework.aigroupId>
    <artifactId>spring-ai-starter-model-openaiartifactId>
dependency>

7.2 Create the Provider Implementation

@Service
@ConditionalOnProperty(name = "spring.ai.openai.api-key")
public class OpenAIChatService implements ChatProvider {

    private final OpenAiChatModel openAiChatModel;

    public OpenAIChatService(OpenAiChatModel openAiChatModel) {
        this.openAiChatModel = openAiChatModel;
    }

    @Override
    public String getChatResponse(String message, String model) {
        // Implementation similar to Ollama
    }

    @Override
    public ProviderType getProviderType() {
        return ProviderType.OPENAI;
    }

    // ... other methods
}

7.3 Add to Provider Type Enum

public enum ProviderType {
    OLLAMA("ollama"),
    OPENAI("openai"),  // Add this
    ANTHROPIC("anthropic"),
    GEMINI("gemini");
    // ... rest of the enum
}

7.4 Configure in application.properties

# OpenAI configuration
spring.ai.openai.api-key=your-api-key-here
spring.ai.openai.chat.options.model=gpt-3.5-turbo

That's it! The provider will automatically be registered and available through the API.

Advanced Features

Error Handling

Our implementation includes comprehensive error handling:

Validation for empty messages
Provider validation
Model validation
Graceful handling of provider errors

Model Selection

Each request can specify a different model:

{
  "message": "Write a poem",
  "provider": "ollama",
  "model": "mistral"
}

Provider Discovery

The /api/providers endpoint allows clients to discover available providers dynamically.

Best Practices

Interface Segregation: The ChatProvider interface is focused and specific
Dependency Injection: Spring manages all dependencies automatically
Error Handling: All errors are handled gracefully with appropriate HTTP status codes
Extensibility: New providers can be added without modifying existing code
Configuration: Each provider can be configured independently

Conclusion

We've built a flexible, extensible AI backend that can work with multiple AI providers. The architecture we've implemented makes it easy to:

Add new AI providers without changing existing code
Switch between providers dynamically
Handle errors gracefully
Validate requests properly
Discover available providers and models

This approach gives you the flexibility to use local models for development and privacy-sensitive applications while being able to switch to cloud providers for production or when more powerful models are needed.

Happy coding! 🚀

Java-Based AI Solutions for Enterprises: Viability, Use Cases, and Market Outlook

DonvitoCodes — Sat, 20 Sep 2025 12:50:20 GMT

Introduction: Java’s Role in Enterprise AI

Java has long been the backbone of enterprise systems, valued for its security, reliability, scalability, and platform independence. These very qualities that made Java the “engine of the enterprise” are now critical for building AI solutions businesses can trust in operations . In fact, enterprises demand production-grade AI that is secure, maintainable, and smoothly integrates with existing systems . This creates a unique opportunity: Java’s mature ecosystem and stability can bridge the gap between cutting-edge AI models and legacy enterprise infrastructure. Modern Java frameworks (e.g. Quarkus, Spring Boot) and new libraries (e.g. LangChain4j, Spring AI) have emerged to simplify integrating large language models (LLMs) and other AI into Java applications . In short, a Java-based AI solution is not only viable – it aligns well with enterprise IT standards and leverages skills many enterprise developers already have. The market timing is ideal too: organizations across the globe are eager to adopt AI, and they are looking for solutions that can plug into their existing workflows with minimal disruption.

Enterprise Needs and AI Adoption Trends

What Enterprises Want from AI: Enterprises are broadly convinced of AI’s potential, but they have clear needs and concerns that any solution must address. Trust, transparency, and governance are top priorities – nearly half of businesses worry about data accuracy and bias in AI outputs . Ensuring AI systems are explainable, fair, and compliant is essential for adoption. Data security and privacy are equally critical: ~40% of organizations cite confidentiality concerns as a barrier, which underscores demand for on-premises or hybrid solutions that keep sensitive data in-house . Data readiness is another need – 42% of enterprises feel they lack sufficient proprietary data to train or customize AI models . This drives interest in solutions that can leverage existing data efficiently (or use techniques like synthetic data generation and federated learning to overcome data gaps ). There’s also a well-documented skills gap: 42% report inadequate in-house AI expertise . Enterprises need higher-level tools and partner support to implement AI without hiring rare (and expensive) talent for every project. Finally, business leaders demand clear ROI and use-case alignment – 42% say they struggle to justify AI projects financially . They seek AI solutions targeting specific pain points (cost reduction, revenue growth, productivity boosts) with measurable outcomes, rather than “AI for AI’s sake.”

Adoption is Accelerating: Despite the challenges, enterprise AI adoption has skyrocketed in the last couple of years. By 2024, nearly 80% of organizations worldwide were engaging with AI in some form – 35% had fully deployed AI in at least one function and another 42% were actively piloting projects . Only a small minority (13%) have no AI plans at all . Surveys also show that AI budgets are swelling. In 2025, 84% of companies planned to increase funding for AI initiatives (up from 73% a year prior) . Large enterprises have moved beyond small pilots – generative AI and ML are now becoming line items in core IT budgets rather than experimental spends . One CIO noted that “what I spent in 2023 I now spend in a week” on AI, reflecting the rapid scale-up of investment . This budget growth is fueled by results: firms are discovering more internal use cases and even deploying customer-facing AI features, which drive tangible business value . Indeed, top-quartile AI adopters report 15–30% improvements in key metrics (productivity, customer satisfaction, etc.) across workflows enhanced by AI .

Global and Regional Momentum: The AI wave is global, with particularly strong momentum in North America and Asia. Asia-Pacific (APAC) enterprises are racing ahead – APAC’s generative AI adoption now ranks just behind North America, and Asian firms are investing very heavily to scale successful use cases . A survey of Asian business leaders found over 90% plan to scale up GenAI projects in the next two years to drive cost savings and revenue gains . Key markets like China and India are championing AI from the C-suite, upskilling their workforce, and aligning AI tightly with business goals . This suggests a huge opportunity in Asian markets for enterprise AI solutions, as organizations there are eager for partnerships to achieve their AI ambitions. (Notably, more than half of APAC companies surveyed plan to work with external partners to expand AI capabilities .)

Demand for Integrated Solutions: Across regions, one clear trend is that enterprises want solutions, not raw tech. A recent report found 98% of executives expect vendors to help shape their AI vision and strategy – yet most feel current vendors fall short in hands-on partnership . Companies don’t just want an AI API; they want guidance, industry-specific insights, and products that seamlessly integrate into their existing processes. In practice, this means a Java AI solution should be packaged in an enterprise-friendly way (discussed more below) and come with support for integration, customization, and governance features out-of-the-box. It should also be flexible deployment-wise, since many firms need to run AI on their own infrastructure for compliance reasons. In short, the market is hungry for robust AI solutions that can be trusted within enterprise environments – and Java, with its enterprise pedigree, is well positioned to deliver on those needs.

AI Use Cases for Enterprises (Text and Vision)

AI can drive value in virtually every industry. Below we highlight key use cases – focusing on Natural Language Processing (NLP) for text-based intelligence and Computer Vision for image/video analysis – that a Java-based AI solution could tackle. These use cases reflect common enterprise pain points that AI is already helping solve:

NLP and Text-Based Use Cases

Enterprise Knowledge Base Q&A: Organizations struggle with information siloed across SharePoint, Confluence, email, etc. Employees waste time searching for answers, and important knowledge is often missed. An AI-powered internal Q&A system can use Retrieval-Augmented Generation (RAG) to answer employees’ questions based on internal documents . This means ingesting and indexing all company docs, and using an LLM to provide natural-language answers with citations. The benefit is faster onboarding and decision-making – turning weeks of digging into a quick chat with an AI assistant. Java integration advantage: Java’s strength in I/O and security is ideal for building the robust pipelines needed to ingest thousands of documents from legacy systems (with proper access controls) . This use case appears in many industries (e.g. a bank’s policy manuals, a manufacturer’s engineering specs, etc.) where unlocking internal knowledge has direct productivity ROI.
Document Data Extraction and Processing: Enterprises drown in unstructured text – contracts, invoices, reports – that require manual data entry or review. AI can automate this. For example, extracting structured data from documents (PDFs, emails, forms) using NLP models can eliminate tedious human entry. One common scenario is processing invoices or purchase orders: an AI model can read incoming documents and pull out key fields (vendor, amounts, line items) for input into ERP systems. This reduces errors and speeds up workflows. Another example is parsing legal documents or contracts to identify clauses and compliance issues automatically. In finance, there are successful cases of AI reading loan applications or financial statements to assist analysts. Java integration: Tools like DL4J or TensorFlow Java can be used to run document parsing models within a Java service, and results can be mapped directly into Java objects (POJOs) for downstream processing . This makes it easy to plug AI-extraction into existing Java-based workflow systems.
Customer Support Chatbots and Virtual Agents: NLP-driven chatbots are a popular AI entry point for many enterprises. These range from simple FAQ bots to advanced virtual agents that handle customer inquiries, IT helpdesk tasks, or HR queries. Modern generative AI can produce far more fluent and context-aware responses than the chatbots of old. For instance, a retail company might deploy an AI chat agent to handle order tracking questions, returns, and product FAQs across web and mobile channels. AI agents can also escalate to human reps when needed, creating a hybrid customer service model. The value is 24/7 support, faster response times, and lower call center workloads. In fact, NLP chatbots can automate ~35% of routine support tasks, leading to substantial cost savings . Java integration: Enterprises often want chatbot systems integrated with their backend data (order databases, CRM, etc.). A Java-based AI solution can shine here by securely connecting the AI to internal systems. For example, one approach is using an agent with tools design: expose certain Java backend services (like “lookup order status”) as tools the AI can invoke . This was identified as a key use case in one Java AI guide – enabling an LLM-powered support agent to not just answer queries, but perform actions like checking account balances via existing Java APIs . Java’s strong security and type safety help ensure the AI only performs allowed actions. Many companies (banks, telcos, e-commerce) are already integrating AI assistants this way to improve customer service while keeping systems secure.
Content Generation and Summarization: Textual content is everywhere in enterprise: marketing copy, internal reports, research summaries, product descriptions, and more. Generative AI (like GPT-style models) provides a way to automate content creation or assist humans with first drafts. Use cases include generating personalized marketing emails, writing knowledge base articles based on bullet points, or creating product descriptions at scale. Summarization is equally valuable – e.g. automatically summarizing lengthy reports or meeting transcripts into concise briefs. This is especially useful in industries like consulting (summarizing research for clients) or media (summarizing news). Many companies are already leveraging AI writers to save time; for instance, Salesforce has employed an internal generative AI (via a platform called Writer) for content and saw four times the expected ROI, including ~$700k saved in the first year . Java integration: A Java AI solution might expose a content generation API (REST endpoints) that internal applications can call, or even integrate into content management systems. With frameworks like Quarkus enabling fast, scalable Java microservices, one can deploy robust content generation services that utilize pre-trained language models (possibly via Java bindings to an OpenAI or local model). Ensuring the generation is “grounded” in enterprise data (via RAG or templates) will be important for accuracy . Because Java is highly scalable, a Java service can handle high-throughput content requests in production – e.g. generating thousands of product summaries for an e-commerce site overnight.
Sentiment Analysis and Voice of Customer: Understanding customer sentiment at scale is another text AI use case. This could involve analyzing customer feedback from surveys, social media, support tickets, or reviews using NLP. For example, a bank might analyze millions of customer feedback comments to detect common pain points or track sentiment trends after a new fee is introduced. AI-driven sentiment analysis can categorize feedback (positive, negative, neutral) and even extract themes (e.g. “customer service wait time” complaints). This helps businesses respond faster to issues and prioritize improvements. Java integration: Sentiment models (like BERT-based classifiers) can be deployed within Java applications to automatically tag incoming text (emails, chats) and route or log them appropriately. Elder Moraes notes that integrating such customer sentiment analysis into high-throughput data pipelines is feasible with Java, outputting results as structured Java objects that downstream systems can use . Many enterprises already use Java for stream processing (e.g. Apache Kafka Streams), and AI can be inserted into those streams for real-time analytics.
Natural Language to Data Query (NL2SQL/NL2API): A growing use case is letting business users ask questions in plain English and have the AI retrieve data or insights from company databases/reports. Essentially, AI as a semantic layer on enterprise data. For instance, a sales manager might ask, “Which product had the highest growth in Asia last quarter?” and the AI would translate that into a secure database query or API call, then return the result. This empowers non-technical staff to get insights without always requiring a data analyst. Several tools and libraries (including some in Java) are emerging to translate NL to SQL. Java integration: Java-based logic can govern this process to ensure security and performance. One reference architecture uses an LLM to draft a SQL query which is then validated by Java code against allowed schemas before execution . This prevents the AI from running unsafe or overly expensive queries. For example, LangChain4j provides patterns for NL-to-API translation that a Java app can implement with type safety. Many industries can benefit: in finance, analysts could query metrics; in retail, regional managers could ask about inventory levels; in healthcare, researchers could query patient stats – all in natural language.
Automated Compliance and Document Review: In heavily regulated sectors (finance, healthcare, legal), firms spend enormous effort reviewing texts for compliance (regulatory filings, contracts, policies). AI can assist by reading and flagging documents that contain certain risks or non-compliant language. For example, an AI system could scan insurance claims and highlight those that possibly violate compliance rules, or compare contract clauses against a company’s standard terms. A case in point: some banks use NLP to analyze communications for compliance breaches. AI can dramatically cut the manual workload here. Java integration: A Java-based solution could incorporate libraries for text analysis to check documents against rulesets. One of the ten use cases described for Java developers is automating regulatory compliance monitoring – using AI to parse legal text and compare it to internal policies . The advantage of Java is that existing compliance systems (often Java-based) can call these AI modules, and the AI can output results into familiar formats (reports, case management queues, etc.). This way, companies can trust the AI as part of their established compliance workflow. Moreover, running such solutions on-premises (which Java facilitates) is often non-negotiable in these industries due to data sensitivity.

(The above are just a selection of NLP use cases. Others include AI-assisted coding (e.g. AI helping developers in writing Java code – IBM’s watsonx Code Assistant is an example targeted at Java modernization), language translation for global companies, HR resume screening, and more. The unifying theme is that enterprises are applying text analytics and generation wherever it automates routine cognitive work.)

Computer Vision Use Cases

Video Surveillance and Security Analytics: Many enterprises use CCTV and surveillance cameras for security, especially in retail, hospitality, transportation, and public sector. AI-based video analytics can automatically detect unusual activities, safety violations, or security threats in real time. During the pandemic, companies accelerated investment in smart surveillance – for example, to ensure social distancing or mask compliance via cameras. Today, AI-enabled video systems can recognize patterns and alert authorities to issues (e.g. an intruder in a restricted area or an unattended bag in an airport) . They can also reduce false alarms compared to rule-based systems by using object detection and behavior analysis. Java integration: Deploying vision models at the edge or within a security software suite can be done with Java (using wrappers for libraries like OpenCV or deep learning models). The challenge is processing massive real-time video feeds, which requires optimization and scalability. Solutions often involve Java for orchestrating streams and using native code (GPU) for heavy vision tasks. Enterprises may prefer on-premise processing for security video (for privacy and low latency). A Java solution packaged to run on edge servers (with GPU support) could address this use case.
Quality Control in Manufacturing: Computer vision for quality inspection is a game-changer in manufacturing. Cameras on production lines can use AI to spot defects or anomalies in products far faster and more consistently than human inspectors. For instance, an AI vision system can examine circuit boards for soldering defects, or food products for packaging flaws, ejecting any defective item immediately. This improves quality and reduces waste. Some factories have reported significant reduction in defect rates by deploying AI vision systems. Similarly, safety compliance can be monitored – e.g. checking if workers are wearing helmets and vests via vision. Java integration: Industrial companies often use Java-based SCADA or MES systems. An AI vision module (perhaps using a JNI bridge to a CNN model) can integrate with these systems to provide alerts and stop machinery when a defect is detected. The predictive maintenance use case is closely related: by visually monitoring equipment (thermal cameras, etc.), AI can predict failures. In fact, AI-driven predictive maintenance (not only vision, but also sensor data) has cut downtime for manufacturers – these systems predict faults before they happen. According to one case, AI-based predictive maintenance helped achieve a 22% boost in efficiency in supply chain operations when combined with overall optimizations . Java’s strength in handling IoT data streams and connecting to databases can complement the AI, storing results and triggering maintenance orders.
Medical Imaging and Diagnostics: In healthcare, AI vision systems are assisting doctors in analyzing medical images like X-rays, MRIs, CT scans, and even pathology slides. This is one of the most impactful AI use cases: models can detect diseases (cancers, retinal diseases, etc.) with high accuracy. For example, a nature study showed an AI model (developed by Google Health) could detect breast cancer in mammograms with 94.6% sensitivity, outperforming expert radiologists who achieved 88.0% . Such AI can act as a “second reader,” catching things a human might miss and reducing false positives. Hospitals and diagnostics labs are adopting AI to get faster and more accurate readings, which improves patient outcomes. Java integration: While the AI models are often developed in Python, deploying them in a hospital’s IT system could involve Java. Many healthcare software systems (PACS imaging systems, electronic health record systems) have Java components. A Java-based AI solution could provide a service that takes in an imaging study and returns AI analysis (with visual annotations), which the radiologist can view in their normal workflow. Because patient data is highly sensitive, an on-prem solution is usually required – which favors a Java solution running within the hospital’s secure network. Also, Java’s reliability is important here: doctors need consistent, validated results, so the AI service must be robust.
Facial Recognition and Biometric ID: Another vision use case is secure access and identity verification using AI. Companies might use facial recognition for physical building access or to verify identity in online processes (e.g. matching a selfie to an ID document for KYC in banking). AI vision can also detect spoofing attempts (ensuring it’s a live person, not a photo). Some airports, for example, use facial recognition for boarding gate identity checks. While this area raises privacy concerns and is regulated in some regions, it’s being adopted for its efficiency. Java integration: Biometric processing often needs to integrate with existing security systems written in Java (for instance, a Java app might interface with camera systems and employee databases). By embedding facial recognition models (via an SDK) into a Java security application, one can build a complete solution that handles capture, AI analysis, and then triggers an action (like unlocking a door or flagging an anomaly). Given Java’s use in many enterprise identity management systems, offering a Java-friendly AI module for this is advantageous.
Retail Analytics and Personalization: In retail environments (physical and online), vision AI and data AI are combined to enhance customer experience and operations. In stores, cameras plus AI can analyze customer foot traffic patterns, dwell time in aisles, or even customer demographics (age/gender) to better understand behavior. On shelves, image recognition can detect out-of-stock products for immediate restocking. Several large retailers in Asia have piloted cashier-less stores where cameras track what items customers pick (like Amazon Go). Personalization is more data-driven but sometimes uses vision input (e.g. smart mirrors that recommend outfits). According to industry reports, retailers are leveraging AI for personalized recommendations and dynamic pricing based on real-time data . AI can analyze purchase history and browsing to tailor offers for each customer, increasing sales. Integration: Retail IT often runs on Java-based platforms (for inventory, CRM, POS systems). A Java AI solution could feed personalized recommendations into e-commerce platforms or power a dynamic pricing engine using reinforcement learning, all within a Java microservice. For in-store vision, results from AI cameras (people counts, heatmaps) could be consumed by Java analytics dashboards for store managers. Scalability is key here due to peaks (holiday rush); AI solutions may use cloud bursts. Indeed, one solution provider noted that with AI and cloud, retailers can scale personalization systems on demand during peak seasons without overprovisioning .
Autonomous Vehicles and Drones: While more specialized, the transportation sector’s use of vision AI is worth noting. Autonomous vehicles rely heavily on real-time computer vision (object detection, lane detection from cameras, LIDAR processing, etc.). Companies like Tesla, Waymo, etc., have advanced this field, and even traditional automakers are embedding driver-assist AI features (adaptive cruise, collision avoidance). For enterprises, this translates to potential use in logistics (e.g. self-driving delivery robots or AI-assisted driving for fleet trucks). Similarly, drones with AI vision are used for inspecting infrastructure (power lines, pipelines) or agricultural fields. These scenarios require processing sensor data on the fly. Java integration: Much of the core AI here is closer to hardware and often in C++/Python, but Java could be used in management software or cloud platforms that aggregate data from vehicles/drones. For instance, a logistics company might have a Java backend system that receives data from AI-equipped drones inspecting warehouses, then uses that to trigger maintenance orders. So while Java might not run on the drone, it plays a role in the broader enterprise workflow integration.

Bottom line: AI use cases in text and vision are extremely broad – spanning security, customer experience, operations, and decision support. The examples above show that many of these use cases have already proven their value (e.g. >50% improvement in fraud detection accuracy using AI , or significant efficiency gains in supply chains ). A Java-based AI business can aim to deliver pre-built solutions or customizable modules for these high-value applications. By focusing initially on NLP and Vision, one covers a large share of enterprise AI needs: from making sense of documents and conversations to interpreting the physical world via images. Crucially, success stories abound in each area (some outlined in the next section), which helps in convincing enterprise clients that these AI interventions are not science fiction – they are achievable and often have well-documented ROI.

Packaging and Integrating a Java AI Solution for Enterprise

Delivering AI to enterprises isn’t just about model accuracy – packaging and integration make the difference between a pilot and a production deployment. To be enterprise-ready, a Java AI solution should consider the following packaging strategies:

Flexible Deployment (Cloud, On-Premise, Hybrid): Many enterprises, due to data security or latency needs, will insist on on-premise or private cloud deployment for AI solutions. Others may be open to a vendor-hosted cloud service if data is less sensitive. The solution should be designed so it can run in the customer’s preferred environment. Using containerization (Docker/Kubernetes) is now standard to enable this portability. For instance, providing the AI components as container images that can be deployed on a Kubernetes cluster (on cloud or on-prem) gives clients freedom. Red Hat’s OpenShift AI platform emphasizes exactly this: the ability to train and deploy AI/ML workloads on-premises, in any cloud, or at the edge, depending on business requirements . A Java-based solution can leverage the write-once-run-anywhere nature of the JVM – running on a customer’s Linux servers or their preferred cloud VM with equal ease. This addresses enterprise concerns around data confidentiality (keep it on-prem if needed) and regulatory compliance (e.g. ensuring data doesn’t leave a region). In practice, a vendor might offer a managed cloud option for convenience but also an on-prem package for customers that need it.
Microservices and APIs: Modern enterprise architectures favor a microservice approach for new capabilities. The AI solution should be packaged as one or more services with clean APIs (REST/GraphQL or even gRPC) that other systems can call. For example, if selling a “document analysis AI”, it might be a service with an endpoint like /extractData or /summarizeDocument. Java frameworks like Quarkus and Spring Boot are excellent for building such services. In fact, Quarkus allows compiling Java apps to native code for fast startup and low memory, making Java services as nimble as ones written in Go or Node for cloud deployment . By offering API access, you make integration easy – enterprises can call the AI from their existing software, regardless of language, as long as they can make an HTTP call. This is crucial because not all parts of a company’s system will be Java; APIs ensure interoperability.
Java SDK/Library Option: In addition to stand-alone services, providing a Java SDK could be valuable for clients that want to deeply embed AI into their own Java applications. For instance, a bank with an existing Java spring application might prefer a jar library they can import to use AI capabilities locally (especially if real-time latency or offline operation is needed). The emergence of libraries like LangChain4j (Java’s answer to LangChain) shows the demand for Java-native APIs to work with LLMs and AI pipelines . A well-documented SDK would allow enterprise developers to, say, call AIClient.summarize(text) or stream data through an AnomalyDetector class in Java. This packaging gives maximum control to the client’s dev team to integrate AI in a way that feels native to their codebase. It’s also a differentiator vs many AI startups that only offer Python libraries or SaaS APIs – a Java SDK appeals directly to the large population of Java enterprise developers.
Integration Connectors: Beyond core algorithms, a sellable enterprise AI solution should include or support connectors to common enterprise data sources and software. For example, if doing an internal knowledge base Q&A, connectors to SharePoint, Confluence, Google Drive, etc., to fetch documents would be needed. If doing vision, connectors to camera systems or an IIoT platform might be relevant. Packaging these integrations (or at least providing them as optional modules) greatly eases adoption. Many enterprise software providers succeed by offering a suite of connectors out-of-the-box. As a Java solution, one can utilize the rich ecosystem of Java connectors (JDBC for databases, JMS for messaging systems, etc.). Java’s longevity in enterprise means there are existing libraries for connecting to almost anything – reusing those can speed up development of integration features. This addresses the common enterprise silo issue: the AI won’t be useful if it can’t pull data from where it lives and push insights to where decisions are made.
Security, Access Control, and Governance: Enterprises will scrutinize the AI solution’s security. Packaging should include robust authentication and authorization mechanisms (integration with LDAP/AD for user auth, role-based access controls for features, audit logging of AI usage, etc.). For example, if offering an AI API, ensure it supports OAuth or integration with the client’s SSO. Java’s security APIs and enterprise frameworks (like Spring Security) can be leveraged to embed these controls. Governance features might include the ability to log and review AI decisions – e.g. logging the input and output of models for audit, or providing an admin dashboard to trace how the AI arrived at an answer (to the extent possible). These needs tie back to the trust issue: features that promote transparency and oversight will make enterprises more comfortable using AI for important tasks. Therefore, the solution might be packaged with an admin UI or logging service that IT and compliance teams can use to monitor AI activity. Since Java is often used for building internal tooling, it’s straightforward to add modules for these governance aspects (for instance, using a Java web framework to create an admin console).
Support for Monitoring and Scaling: In production, enterprises will expect the AI solution to have monitoring hooks, performance dashboards, and scaling options. Container orchestration (K8s) will handle some scaling, but the solution should expose metrics (via JMX or Prometheus endpoints, for example) so that the company can monitor throughput, latency, error rates of AI components. Packaging should thus include metric instrumentation (e.g. using Micrometer in Spring) and possibly autoscaling guidelines or Helm charts if Kubernetes is used. Many enterprise Java apps use APM tools (like New Relic, AppDynamics); providing compatibility with those for the AI service is a plus. Essentially, treat the AI solution like any enterprise application: it must fit into the IT department’s operational ecosystem. Java’s advantage here is that ops teams are very familiar with running and tuning Java services – the GC, thread pools, etc. – so they can apply the same rigor to the AI service as to other Java apps. This familiarity can reduce the friction of deploying an AI solution (versus, say, a Python-based service which might require different monitoring setups).
Partner-Like Delivery: As noted, enterprises want vendors who will partner in their AI journey . So beyond the technical packaging, consider packaging the solution as a service offering with consulting/training. For example, a business plan might include offering professional services: helping customize models for the client’s domain (fine-tuning an NLP model on their data), training their staff on using the AI tool, and co-developing new use case extensions. Many enterprises are seeking guidance on AI strategy – packaging your solution with an “AI integration workshop” or pilot program can differentiate it. This isn’t a software packaging per se, but a go-to-market packaging that aligns with enterprise expectations. It can increase trust that your Java AI solution isn’t a black box you drop off, but rather a living product that will be integrated hand-in-hand with their team.

In summary, packaging for enterprise means making deployment and integration as easy as possible within existing IT frameworks. Containerized microservices, Java APIs, connectors, security and monitoring features – these ensure the AI solution can be adopted without weeks of re-engineering the client’s environment. Java is a strong choice for implementing all of this due to its enterprise toolchain maturity. A concrete example: IBM’s own AI offerings (like Watson) often provided on-premise Java-based packages for banks that needed to run AI in-house, showing that this model is feasible and in demand. Similarly, frameworks like Red Hat OpenShift AI highlight that enterprises want platforms that bring data scientists and Java developers onto one common platform with consistency in deployment and governance . Your Java solution can ride this wave by presenting itself as an “enterprise AI platform/module” rather than just an algorithm – effectively lowering the barrier for companies to plug AI into their operations.

Success Stories and Case Studies

It’s important for a business plan to highlight what has already been achieved with AI in enterprises – both to validate the concept and to provide learning from successes. Here we compile a few representative success stories and studies across industries (global and Asian) that showcase enterprise AI integration, many of which could be delivered or enhanced by a Java-centric approach:

Finance (Fraud Detection): Financial services have embraced AI to combat fraud in transactions. A Forbes analysis found that AI-based systems improved fraud detection accuracy by over 50% compared to traditional rule-based methods . For example, major credit card networks use machine learning models to analyze millions of transactions in real time, flagging anomalies that humans might miss. These AI systems have saved banks hundreds of millions by preventing fraudulent charges. Notably, they must integrate with legacy banking systems (often written in COBOL or Java). Firms like JP Morgan also leverage AI for other tasks – e.g. AI document review (their COIN tool reportedly saved 360k hours of legal work by auto-reviewing loan documents). Many banks run Java-heavy IT stacks, so incorporating AI via Java services was a natural path. The success in fraud detection is an ideal proof point for AI’s ROI – reducing losses and operational costs while improving security.
Healthcare (Diagnostics and Operations): We already mentioned AI’s performance in medical imaging. To add: deployments of AI in radiology at hospitals in the US, Europe, and Asia have shown measurable impact. For instance, a large hospital in India using an AI chest X-ray tool saw a significant increase in tuberculosis detection in early stages, enabling earlier treatment. A UK hospital network using an AI stroke detection on brain scans managed to cut average diagnosis time from 3 hours to 15 minutes. These are life-saving improvements. A study in Nature demonstrated an AI model exceeding radiologist accuracy in breast cancer detection (94.6% vs 88.0% sensitivity) , suggesting AI can augment even expert professionals. Beyond imaging, hospitals use AI (often via natural language processing) to optimize operations – e.g. predicting patient no-shows, automating appointment scheduling via chatbot, and transcribing doctors’ notes. One success story is Vizient (US), a healthcare performance improvement company: they partnered with an enterprise AI vendor (Writer) and achieved four times the estimated ROI on generative AI applications, saving about $700k in the first year while empowering employees across departments to use AI in their workflows . This underscores that with the right strategy and tools, AI can quickly pay for itself in healthcare administration. Java comes into play since many hospital IT systems (appointment systems, EHR integration layers) are Java-based – the AI solutions often had to integrate seamlessly, which they did, proving the viability of adding AI without ripping out existing systems.
Manufacturing (Industry 4.0): In manufacturing and supply chain, there are numerous case studies. Siemens, for example, implemented AI-driven predictive maintenance across its factories, which reduced unplanned downtime by 20-30% on certain production lines. Bosch applied computer vision in visual inspection and reported higher defect capture rates, improving product quality with minimal increase in inspection time. A Capgemini study noted that 68% of supply chain organizations have implemented AI-based traceability/visibility solutions, leading to an average 22% increase in efficiency in those operations . One success story in Asia is a Japanese automotive manufacturer that used AI vision to detect tiny paint imperfections on car bodies – something previously done by human inspectors – and they achieved near-zero customer complaints about paint quality after deployment. These successes often involve AI alongside IoT (sensors, cameras) in an “Industry 4.0” paradigm, and integration with MES/SCADA systems (where Java is common) was key. The overall potential savings in manufacturing from AI (in maintenance, yield improvement, inventory optimization) run into billions globally, and leading firms have demonstrated these gains in practice, setting a blueprint for others.
Retail and Customer Experience: Retailers have seen both top-line and bottom-line benefits from AI. For example, Amazon’s recommendation engine (an early AI success) drives an estimated 30% of its e-commerce revenue through personalized suggestions. Brick-and-mortar retailers in Asia (like Alibaba’s Hema supermarkets in China) use AI to manage inventory and logistics in real time, ensuring fresh products and efficient supply. Walmart uses AI for optimizing store layouts and on-shelf availability (even using robots for shelf scanning). On the customer front, many retailers and telecom companies implemented AI chatbots to handle support – Vodafone deployed a customer service bot (“TOBi”) which resolved large volumes of inquiries, leading to higher customer satisfaction and millions in annual savings. These cases show that AI can improve customer experience while lowering service costs. Importantly, companies like Salesforce have integrated generative AI features into their CRM products, and clients (like Mars and Accenture) are using those to auto-generate marketing content or sales emails. In one anecdote, Salesforce internally empowered 50+ “AI champions” to build AI apps for their workflows using a low-code AI studio. That cultural approach, alongside technology, led to high adoption internally. From a Java perspective: many e-commerce platforms (like those using SAP Hybris or custom Java eCommerce engines) have integrated AI modules for recommendations and search (often via Java-based microservices calling ML models). The success here is measured in improved conversion rates and basket sizes – and several studies show personalized recommendations can boost sales by 5-15%. Given retail’s thin margins, that’s significant.
Public Sector and Others: AI success stories aren’t limited to profit-driven companies. Government agencies have used NLP to automate paperwork (e.g. an Australian agency using AI to process citizen emails saw response times drop from weeks to hours). In Singapore, the government deployed computer vision AI to monitor and predict traffic conditions, which improved traffic flow and informed infrastructure planning (part of the “smart nation” initiative). In agriculture, companies use vision AI via drones to monitor crop health over large areas, increasing yields by identifying issues early (pilots in India and Africa have shown crop yield improvements of ~15% by targeted interventions guided by AI analysis). Even in legal services, startups like Harvey (built on GPT) have shown that AI copilots can save lawyers significant time on research and drafting – some law firms report 20-50% time savings on certain tasks using these tools. Many of these emerging applications are in their early stages but have had convincing pilot results. The pattern across all these is clear: when AI is applied thoughtfully to a well-defined use case, it delivers efficiency, accuracy, or scalability improvements that were hard to achieve otherwise.

It’s also instructive to note why some AI pilots succeeded where others failed. Common success factors include: executive sponsorship and alignment with business goals (BCG found Asian companies with CEO-level AI sponsorship scaled AI more successfully) ; starting with a high-impact use case (rather than generic experiments); ensuring data quality; and having cross-functional teams (IT working with business units) drive the project. These lessons inform the approach of any AI solution provider – it’s not just technology, but also enabling the client to adopt it properly. As a vendor of a Java AI solution, highlighting these success stories helps convince potential enterprise customers that “others in your industry have done this and got results – we can help you do the same.” It also helps in identifying which use cases are ripe for quick wins (e.g. if a prospect is a bank, pointing to fraud detection and customer chatbot successes; if manufacturing, pointing to predictive maintenance and QA examples).

Future Outlook and Opportunities

The potential for AI in enterprise is still expanding rapidly. By focusing on Java-based AI integration, we tap into a future where AI becomes ubiquitous across business functions – and Java acts as the reliable conduit for that ubiquity. Here are some forward-looking points and opportunities:

AI Becoming Standard in Software: We are nearing a point where most enterprise software will have AI features baked in. Over 60% of enterprise SaaS products already have embedded AI capabilities as of 2025 , and that proportion will grow. This means enterprises will expect any new solution to be “AI-enabled” or at least AI-ready. Offering a Java platform that can “AI-ify” their existing systems is a huge opportunity. Just as databases became a standard component of software in the 2000s, AI services (for prediction, classification, generation) could become a standard component in the 2020s. A Java AI business can position itself as the provider of that standard layer for companies that have a lot of Java systems.
Generative AI and Multi-Modal Fusion: The rise of large language models and generative AI opens new use cases that barely existed a couple years ago – from AI code assistants to creative content generation to conversational analytics. Enterprises are exploring “copilots” in every department (marketing, sales, finance, HR, engineering) , essentially AI assistants tailored to specific jobs. For example, an HR copilot might help draft job descriptions and screen resumes; a finance copilot might auto-generate budget reports and answer queries about financial data. These copilots need to interface with internal data securely – again a strength of an on-prem or tightly integrated Java solution. Moreover, the future is multi-modal AI: combining text, vision, audio, and structured data. Many enterprise tasks involve multiple data types (e.g. a maintenance AI might take text logs, sensor readings, and images of equipment). The potential is in fusing these modalities to provide deeper insights. A Java-based platform that can orchestrate various AI models (an image model, a text model, etc.) into one workflow would be ahead of the curve. We already see early adoption of this in, say, insurance: when processing a car accident claim, AI can analyze images of the damage, read the text description, and cross-check with historical claim data – all to automate payout decisions. The tools to do this are emerging, and a cohesive solution in Java could find a strong market.
Agentic AI and Automation of Processes: Looking a bit further, enterprises will likely move beyond single-task AI to AI agents that can handle sequences of tasks and autonomously interact with systems. For instance, instead of just answering a question, an AI agent might be told, “ensure our website is up to date with the latest product info,” and then it will gather info, log into the CMS, update pages, and so forth by calling APIs – under some human supervision. This kind of autonomous workflow execution (sometimes called agentic AI) is on the horizon. Enterprises running on Java backends will need their AI agents to talk to Java systems. Already, frameworks like LangChain allow AI to invoke tools; a Java equivalent (LangChain4j, etc.) is bringing that capability to the Java world . The potential here is automating a lot of routine “digital” tasks via AI. Companies that crack this (with guardrails) could see massive efficiency gains. It’s an area to watch and incorporate into the product roadmap.
Increasing AI Adoption in Asia: The Asian market specifically shows signs of faster adoption and scaling in the next few years. APAC companies are not just piloting but seriously scaling AI – many expect substantial boosts in revenue and efficiency. In 2025, APAC firms that have scaled GenAI already reported 25% shorter time-to-market for new products on average . Countries like Singapore, China, and India have national AI strategies pushing enterprise adoption. This suggests that any AI solution business should have a strategy for Asia – perhaps local partnerships or ensuring the solution supports local languages (for NLP) and local cloud infrastructure. Java’s wide use in Asia (especially India’s IT sector and many ASEAN enterprises) is a plus for market entry. The potential is a rapidly growing client base in Asia that could even leapfrog Western companies in AI integration, as they sometimes build new systems with AI at the core rather than as an add-on.
Continuous Market Growth: All indicators show the enterprise AI market will continue its explosive growth. The global AI market size, estimated around $390+ billion in 2025, is projected to reach $1.8 trillion by 2030 (nearly 35% CAGR) . This growth will be driven by both deeper penetration in existing use cases and expansion to new frontiers. For instance, small and mid-size enterprises (SMEs) are behind large firms in AI adoption, but as solutions become more turnkey, SMEs represent a large untapped segment. A Java-based solution could be packaged in a more accessible way for mid-size companies (who also often run on Java systems like ERP packages). Also, public sector and education could become big consumers of AI solutions (with government often preferring vendors that support on-prem and open standards – again, an edge for Java solutions).
Competitive Landscape and Differentiation: While the opportunity is vast, competition is also growing. Tech giants offer AI cloud services (AWS, Azure, GCP) and specialized startups pop up weekly. However, there is a potential gap in enterprise-specific, integration-focused offerings. Many startups provide flashy AI demos but not the hard integration work. Enterprises often struggle to operationalize those. This is where a Java enterprise AI business can differentiate: by emphasizing integration, customization, and trust. Essentially, “we don’t just have a great model, we make it work in your environment safely and reliably.” Over 94% of executives are not completely satisfied with their current AI vendors – often because of poor integration or support. That means a space for new players who get the enterprise mindset. The future likely will see consolidation where enterprises choose a few core AI platforms to standardize on (similar to how they chose databases or ERP systems). Positioning to become one of those standard platforms – especially leveraging Java’s huge installed base – is a huge potential win.

In conclusion, the viability of a Java-centric AI solution for enterprises is strongly supported by current trends and future directions. Enterprises need AI, but they need it on their terms – integrated with existing systems, governed properly, and delivering clear business value. Java’s ecosystem, performance, and enterprise adoption make it an excellent foundation to meet these needs. The use cases in text and vision we discussed are just the beginning; as AI technology advances (with better models and techniques), a Java-based platform can continuously incorporate those advances and offer them in an enterprise-friendly package. The market is validating the concept with ever-increasing adoption rates and success stories. With thoughtful packaging, strong domain use cases, and attention to enterprise requirements, a Java AI business can position itself as a key enabler in the AI transformation of global and Asian enterprises – a transformation that is well underway and poised to accelerate through 2025 and beyond.

Sources:

Elder Moraes, 10 Enterprise AI Use Cases Java Developers Are Ready To Build Today
IBM, The 5 Biggest AI Adoption Challenges for 2025 (survey data on enterprise AI hurdles)
A16Z, How 100 Enterprise CIOs Are Building and Buying Gen AI in 2025
BCG, In the Race to Adopt AI, Asia-Pacific Is the Region to Watch
InfoQ Panel, AI Integration for Java – to the Future, from the Past (Java ecosystem tools)
Hyperstack/NexGen, Top 10 AI Use Cases in Enterprise 2025 (industry examples)
Hyperstack/NexGen (Case studies: Fraud detection +50% accuracy ; Medical AI vs radiologists ; Supply chain stat )
Writer.com, 2025 Enterprise AI Adoption Report (organizational aspects and ROI)
Founders Forum, AI Statistics 2024–2025 (global adoption and market size)
Red Hat, OpenShift AI Overview (packaging AI for hybrid cloud) .

9 Ways How AI can be Useful for Emails

DonvitoCodes — Tue, 26 Aug 2025 04:54:39 GMT

Email is essential for modern communication, but it often takes up more time than it should. Crafting thoughtful replies, keeping the right tone, and managing dozens of threads can quickly become overwhelming. This is where AI can step in and make a real difference.

AI can act as your writing partner for drafting and polishing replies so you spend less time typing and more time focusing on meaningful work.

Below, we’ll explore the key ways AI can help with replying to emails, with some examples.

1. Drafting Replies Quickly

One of the most immediate benefits of AI is speed. You can feed it the main idea of what you want to say, and it can generate a professional draft almost instantly.

Example:

Your input:

Tell client Sarah we got the report and will review by Friday.

AI’s draft:

“Hi Sarah,

Thank you for sending over the report. Our team will review it and share our feedback with you by Friday. 

Best regards, 

Melvin.”

2. Adapting Tone and Style

Different situations require different tones. A reply to your boss might need to be formal, while a note to a close colleague can be casual. AI can instantly adapt.

Example:

Formal:

“Dear Mr. Lee, I appreciate your update. I will review the materials and revert with feedback by the end of this week.”

Friendly:

“Thanks for the update, Lee! I’ll go through the docs and get back to you by Friday.”

You get flexibility without extra effort.

3. Handling Different Types of Emails

Not every email needs the same depth. AI can handle quick acknowledgments, detailed breakdowns, or polished high-stakes responses.

Quick acknowledgments

“Got it, thanks for the heads-up!”

Detailed explanations

“Hi team, here’s a breakdown of the next steps: (1) finalize the draft, (2) align with marketing, and (3) send for client approval by Thursday.”

Polished responses for sensitive topics:

“I understand your concerns. Let’s schedule a call this week to walk through the details together and ensure we’re fully aligned.”

AI can shift gears depending on the situation, something that usually takes us more mental energy than we realize.

4. Saving Time With Long Emails

We’ve all faced those long emails packed with multiple questions or concerns. Writing a structured reply can be exhausting. AI can break the problem down and generate organized responses.

Example:

Input: A client sends a 10-paragraph email with five separate questions

A client sends a 10-paragraph email with five separate questions.

AI output:

Opens with appreciation: “Thank you for the detailed update.”

Breaks down answers point by point in bullet form.

Closes with: “Please let me know if this addresses everything, or if we should set up a call to clarify further.”

This makes your replies not only faster but also clearer.

5. Maintaining Consistency

For businesses, consistency in tone and phrasing matters. AI can be “trained” on preferred phrases, ensuring every reply reflects your voice or brand.

Example:

Instead of writing

“I’ll get back to you soon,”

AI can consistently generate

“We’ll get back to you shortly”

if that’s your company’s standard wording.

Over time, this builds trust and professionalism.

6. Prioritizing Responses

Inbox overload is common. AI can help you decide which emails need immediate attention and even propose quick replies.

Example:

Email:

“Can you confirm the delivery date today?”

AI suggests:

“Hi John, yes—the delivery is scheduled for September 2. Please let me know if you need any changes.”

7. Overcoming Writer’s Block

Sometimes the hardest part of replying to an email is just getting started. AI can help you overcome that problem.

Example:

You’re asked to decline a partnership politely.
AI draft:

“Thank you for reaching out and considering us for this collaboration. At this time, we are not in a position to move forward, but we appreciate your proposal and will keep your details on file for future opportunities.”

This saves you from overthinking sensitive responses.

8. Multilingual Replies

For global teams and clients, AI can generate or translate replies into different languages—while keeping the professional tone intact.

Example:

Input:

“Tell partner in French we’ll share the final report tomorrow.”

AI output:

“Bonjour Marie, nous partagerons le rapport final demain. Merci pour votre patience.”

This makes international communication smoother and faster.

9. Customizing Length and Detail

Sometimes you want short replies, other times a more detailed response. AI lets you control the length and depth.

Example:

Brief:

“Thanks for the update, Friday works for me.”

Detailed:

“Hi James, thanks for the update. Friday works well, and I’ll prepare the agenda in advance so we can focus on the new product features during our discussion.”

AI won’t replace your judgment or decision-making, it complements it. Think of it as a reliable writing partner that:

Turns your shorthand into polished messages
Adapts tone to fit the recipient
Handles complex replies with structure
Saves time and reduces inbox stress

I hope this article helped you gave you some understand how you can use AI for emails.

How do you use AI for emails? Feel free to comment below

You can also join my Practical with AI newsletter for more AI https://practicalwithai.substack.com/

Introducing OpenAI o3-mini: A New Frontier in Cost-Effective STEM Reasoning

DonvitoCodes — Sat, 01 Feb 2025 05:55:12 GMT

OpenAI unveiled its latest innovation: OpenAI o3-mini. This new model is a major leap forward in efficient, high-quality reasoning—especially in STEM fields such as science, mathematics, and coding. Today, we explore what makes o3-mini so exciting, its standout performance metrics, and how it is shaping the future of AI-driven reasoning.

A Fresh Approach to Reasoning Models

OpenAI o3-mini is described as “pushing the frontier of cost-effective reasoning” by combining speed with exceptional performance in specialized domains. Unlike previous models, it has been optimized to deliver fast, accurate responses while keeping operational costs low. This makes it particularly attractive for applications where precision in STEM-related tasks is a priority .

Key Innovations

• Cost Efficiency & Speed: With a significant reduction in per-token pricing (a 95% reduction since GPT-4’s launch), o3-mini provides not only rapid responses but also lower latency—averaging 2.5 seconds faster time to first token compared to its predecessor .

• Optimized STEM Reasoning: Designed specifically to tackle challenges in math, science, and coding, o3-mini demonstrates advanced reasoning capabilities, making it ideal for both academic and industrial applications.

• Flexible Reasoning Effort: Developers can choose between three reasoning effort options—low, medium, and high—to best suit the complexity of their use case. This flexibility ensures that o3-mini “thinks harder” when necessary or prioritizes speed when required .

Performance That Speaks for Itself

One of the most compelling aspects of OpenAI o3-mini is its rigorous performance on various benchmarks:

Competition Math (AIME 2024)

• Mathematics: With medium reasoning effort, o3-mini matches the performance of previous models on challenging math questions. When pushed to high effort, it outperforms both OpenAI o1-mini and even its broader general knowledge counterpart .

PhD-level Science Questions (GPQA Diamond)

• Science Excellence: In high-difficulty science challenges, o3-mini proves its mettle by achieving comparable performance to the more extensive OpenAI o1 model when using high reasoning effort. This ensures that users looking for precision in academic or research settings can rely on its output.

Code and Software Engineering

• Competition Coding: On platforms like Codeforces, o3-mini exhibits significant improvement in coding tasks with an Elo rating of 2073 at high reasoning effort.

• Software Engineering Benchmarks: In tests like SWE-bench Verified, o3-mini delivers the highest accuracy among its peers, making it an attractive option for developers involved in production-level coding tasks .

Additional Metrics

• LiveBench Coding: Further tests confirm that even at medium reasoning effort, o3-mini can outperform previous models in speed and overall efficiency.

• General Knowledge: Its comprehensive training allows it to excel not only in technical subjects but also in broader general knowledge areas, ensuring a well-rounded performance across diverse queries.

Empowering Developers with New Features

For the developer community, o3-mini brings several highly anticipated features:

• Function Calling and Structured Outputs: These capabilities allow for seamless integration into production workflows, giving developers the tools needed for more dynamic and structured interactions .

• Developer Messages and Streaming: Out-of-the-box support for streaming ensures that responses are delivered quickly, which is crucial for real-time applications.

• Upgraded API Tiers: With improvements in rate limits—tripling the messages per day for ChatGPT Plus, Team, and Pro users—OpenAI is making it easier for a wide range of users to access and benefit from its advanced reasoning capabilities.

Safety and Responsible Deployment

OpenAI has placed a strong emphasis on safety with o3-mini. By employing techniques such as deliberative alignment, the model is trained to adhere to human-written safety guidelines before responding to prompts. Extensive evaluations—including disallowed content and jailbreak tests—ensure that o3-mini not only performs well but also responds in a safe and controlled manner .

Looking Ahead

The release of OpenAI o3-mini is more than just an upgrade—it’s a demonstration of how targeted innovations in AI can transform the landscape of STEM problem-solving. With its balance of speed, precision, and cost-efficiency, o3-mini is set to become a vital tool for educators, developers, and researchers alike.

In summary, OpenAI o3-mini represents a significant advancement in the realm of AI reasoning models. Its performance across mathematical, scientific, and coding evaluations confirms that small models can indeed achieve high levels of intelligence without compromising on efficiency or safety. As OpenAI continues to refine and expand its models, we can expect even more breakthroughs that will further democratize access to high-quality AI.

For further details on OpenAI o3-mini, including technical benchmarks and full evaluation metrics, visit the official OpenAI page .

My Top AI Coding Tools

DonvitoCodes — Mon, 28 Oct 2024 09:43:15 GMT

AI tools are revolutionizing software development by offering a range of capabilities, from code completion and generation to UI design and code review, with notable tools like GitHub Copilot, Cursor AI, and Vercel v0 providing unique features such as real-time suggestions and AI-powered UI generation. Meanwhile, platforms like Codeium, Continue.dev, and Bolt.new focus on privacy, open-source integration, and web-based coding assistance, catering to diverse developer needs and enhancing productivity across various stages of the development process.

AI Coding Copilots

AI coding copilots have revolutionized software development, offering a range of features to enhance productivity and code quality. Here's a more detailed look at some of the leading AI-powered coding assistants:

Cursor AI
- Built on Visual Studio Code for a familiar interface
- Offers deep AI integration for code generation and refactoring
- Analyzes entire codebases for context-aware suggestions
- Provides in-line documentation and explanations
- Supports natural language prompts for code generation
GitHub Copilot
- Integrates with various IDEs including VS Code, JetBrains, and Vim/Neovim
- Provides real-time code suggestions based on context
- Excels in generating boilerplate code and completing repetitive tasks
- Trained on a vast repository of open-source code
- Offers a chat interface for more complex coding queries
Vercel v0
- Specializes in UI generation, transforming text descriptions into functional React components
- Provides real-time preview of generated UI components
- Supports integration with popular frontend frameworks and libraries
- Offers accessibility considerations in generated components
- Allows for easy deployment and integration with existing projects
Codeium
- Supports 70+ programming languages
- Offers in-line Fill-in-the-Middle technology for improved accuracy
- Emphasizes privacy and security with options for on-premise deployment
- Provides natural language processing for code generation
- Integrates with popular IDEs and code editors
Continue.dev
- Open-source AI coding assistant
- Supports multiple AI models, including local LLMs
- Integrates with existing IDEs for seamless workflow
- Allows querying of API endpoints for flexible AI assistance
- Focuses on customizability and developer control
Supermaven
- Specializes in AI-assisted code review and bug detection
- Helps improve code quality by catching potential issues early
- Offers suggestions for code optimization and best practices
- Integrates with version control systems for streamlined workflows
- Provides detailed explanations for detected issues and suggested fixes
Bolt.new
- Web-based AI coding assistant for rapid prototyping
- Supports natural language prompting for code generation
- Offers multi-framework support for various web technologies
- Provides integrated deployment options
- Features an AI-powered chat interface for coding assistance

These AI copilots cater to different aspects of the development workflow, from code generation and completion to UI design and code review, offering developers a comprehensive toolkit to streamline their coding process and enhance productivity.

My AI Development Stack

I leverage a powerful combination of AI-powered development tools to streamline my coding workflow. Cursor serves as my primary IDE, offering intelligent code suggestions and comprehensive codebase analysis while maintaining a familiar VS Code-like environment.

For rapid prototyping and web development, I use Bolt.new's web-based interface, which excels at quickly turning ideas into functional applications with its natural language prompting and seamless deployment capabilities.

Vercel v0 handles my UI component generation needs, transforming text descriptions into polished React components with built-in accessibility features.

Complementing these, Codeium provides robust code completion across multiple languages, with its Fill-in-the-Middle technology offering accurate suggestions. This combination creates a powerful ecosystem that enhances productivity across different aspects of the development lifecycle, from initial prototyping to polished implementation.

I also regularly chat with Claude AI and ChatGPT when I need help solving tricky coding problems or want to understand complex concepts better. While I need to copy-paste code from these chat AIs, their help with debugging and explaining things clearly makes them essential to my workflow.

Cursor AI for Interative Development

Cursor AI, available at cursor.com, is a cutting-edge AI-powered code editor that has gained significant traction among developers for its innovative features and seamless integration of artificial intelligence into the coding process.

Built on top of Visual Studio Code, Cursor AI combines the familiarity of a traditional IDE with advanced AI capabilities, offering a unique coding experience. The editor provides context-aware code suggestions and completions, significantly enhancing developer productivity. It can generate entire functions, classes, and code blocks based on natural language prompts, allowing developers to express their ideas in plain English and have them translated into functional code.

One of Cursor AI's standout features is its ability to understand and analyze the entire codebase, providing suggestions and insights that are relevant to the project as a whole, not just the current file. This comprehensive understanding enables more accurate and contextually appropriate code generation and refactoring suggestions.

Cursor AI also excels in code documentation and explanation. It can automatically generate meaningful comments and documentation for existing code, saving developers significant time and improving code maintainability. Additionally, it offers an integrated AI chat feature that allows developers to ask questions about their code or request explanations for complex functions without leaving the editor.

The platform prioritizes privacy and security, offering a privacy mode where none of the user's code is stored by the company. This feature, along with its SOC 2 certification, makes Cursor AI a viable option for developers working on sensitive projects or within organizations with strict data protection requirements

Cursor AI offers a free 2-week trial with full-features. Paid plans starting at $20 per month for individuals and $40 per month for businesses. The business plan includes additional features such as organization-wide enforced privacy mode and an admin dashboard for project management You can also use your own OpenAI or Anthropic Claude API key with Cursor.

Bolt.new - a great starting point!

Bolt.new is a powerful web-based AI coding assistant that stands out among other AI tools for software development. Unlike traditional IDEs or code editors, Bolt.new operates entirely in the browser, offering a unique and accessible approach to AI-assisted coding1.Key features of Bolt.new include:

Rapid prototyping: Bolt.new excels at quickly turning ideas into functional web applications, allowing developers to create and deploy projects in minutes.
Natural language prompting: Users can describe their desired application in plain English, and Bolt.new will generate the corresponding code and project structure1.
Multi-framework support: Bolt.new can work with various web frameworks, including Next.js, React, and others, adapting to different project requirements.
Integrated deployment: The platform offers one-click deployment to Netlify, streamlining the process of making applications live on the internet.
AI-powered chat: With the recent addition of "Ask Bolt," developers can now interact with AI directly within the editor, enhancing the coding experience.
Code generation and completion: Bolt.new provides context-aware code suggestions and can generate entire functions or components based on user prompts.
Beginner-friendly interface: The platform's UI is designed to be accessible to newcomers, resembling familiar chat interfaces like ChatGPT or Claude.

Bolt.new's web-based nature sets it apart from standalone editors like Cursor AI or IDE plugins like GitHub Copilot. It offers a more integrated experience, handling everything from initial code generation to deployment within a single platform. This makes it particularly useful for rapid prototyping and idea validation, allowing developers to quickly bring concepts to life without the need for extensive setup or local development environments.

Vercel v0 - Generate UI Components

Vercel v0 is a groundbreaking AI-powered UI generation tool that transforms natural language descriptions into functional React code. Launched by Vercel, v0 streamlines the process of creating user interfaces by leveraging generative AI technology to produce high-quality, customizable components based on shadcn/ui.Key features of Vercel v0 include:

Text-to-UI conversion: Users can describe their desired interface in plain English, and v0 generates corresponding React components.
Real-time preview: As code is generated, v0 provides an instant visual representation of the UI, allowing for immediate feedback and iteration4.
Code customization: Developers can refine and modify the generated code directly within the v0 interface.
Integration with popular frameworks: v0 produces code compatible with React and Next.js, making it easy to incorporate into existing projects.
Support for third-party libraries: The tool can integrate components from various libraries, including Material UI and react-three-fiber for 3D graphics.
Accessibility considerations: v0 generates components with built-in accessibility features, adhering to WAI-ARIA guidelines.
Deployment options: Users can copy the generated code or install it directly into their codebase using the shadcn CLI.

v0 excels in rapid prototyping and UI experimentation, allowing developers and designers to quickly iterate on ideas without writing extensive code from scratch. It's particularly useful for creating landing pages, pricing tables, and other common web components.While v0 is primarily focused on frontend development, it also offers some backend integration capabilities, such as fetching data from external sources.

The tool is continually evolving, with plans to add support for custom design systems, theming, and image-to-code transformations.Vercel v0 is available in both free and paid tiers, with subscription plans offering increased generation credits and additional features. As the tool transitions from Alpha to Beta, it's gaining popularity among developers for its ability to accelerate the UI design and development process

Codeium AI Code Completion

Codeium is a powerful AI-powered code completion tool that offers functionality similar to GitHub Copilot, but with some unique features and advantages. It provides context-aware suggestions and multi-line code generation across 70+ programming languages. Codeium's autocomplete feature is designed to be faster than thought, producing high-quality suggestions with incredibly low latencies.Key features of Codeium include:

In-line Fill-in-the-Middle (FIM) technology, which improves the accuracy of code completion by considering surrounding context.
Integration with popular IDEs and code editors such as Visual Studio Code, IntelliJ, Sublime Text, and more.
A free version for individual developers, with paid plans starting at $12 per user per month for additional features.
Privacy-focused approach, analyzing only necessary code and offering options for on-premise deployment.
Natural language processing capabilities, allowing developers to turn plain text prompts into code4.

Unlike GitHub Copilot, Codeium emphasizes its commitment to user privacy and offers flexible deployment options, making it an attractive alternative for developers and organizations with specific security requirements.

Using Claude and ChatGPT for Coding

Claude AI and ChatGPT are powerful AI chat tools that can assist with coding tasks, but they require manual code transfer. Unlike integrated development environments (IDEs) with AI plugins, these models operate through chat interfaces, necessitating a copy-paste workflow for code implementation. When using Claude or ChatGPT for coding:

Describe your coding task or problem in natural language, providing context and specific requirements.
The AI will generate code snippets or explanations based on your prompt.
Manually copy the generated code from the chat interface.
Paste the code into your preferred IDE or code editor for testing and integration.
Iterate by asking follow-up questions or requesting modifications to refine the code.

This process allows for flexible problem-solving and learning, as you can engage in a dialogue about the code, ask for explanations, or request optimizations. However, it requires more manual effort compared to integrated AI coding assistants. Users must carefully review and test the pasted code, as these models may occasionally produce errors or outdated syntax.

Despite this limitation, ChatGPT and Claude Sonnet offer valuable coding assistance, especially for conceptual understanding, algorithm design, and troubleshooting complex issues. I rely on this whenever I wanted to understand concepts and generate sample code for m problem.

Claude's Artifacts Feature

Claude AI's Artifacts feature represents a significant advancement in AI-assisted coding and data visualization. Artifacts allow Claude to generate and display substantial, standalone content in a dedicated window separate from the main conversation. This feature is particularly useful for coding tasks, as it enables real-time visualization and iteration of code snippets, documents, and even interactive components.

For coding, Artifacts can display various types of content:

Code snippets in multiple programming languages
HTML webpages with integrated CSS and JavaScript
SVG (Scalable Vector Graphics) images
Mermaid diagrams for visualizing workflows and processes
React components that run directly in the browser

Claude's ability to generate and manipulate charts and graphs within Artifacts is especially noteworthy. Users can request data visualizations, and Claude will create interactive charts using libraries like Recharts. This capability extends to creating infographics and other visual representations of data, making it a powerful tool for data analysis and presentation.

The Artifacts feature also supports version control, allowing users to switch between different iterations of the generated content. This functionality, combined with Claude's ability to understand and analyze entire codebases, makes it a versatile assistant for complex coding tasks and data-driven projects.

The Future of Development

Today's AI tools have transformed how I write code. My daily workflow combines specialized coding assistants that help me write, analyze, and improve code with great efficiency. From intelligent code suggestions in my editor to rapid prototyping and UI generation, these tools handle the heavy lifting of routine tasks. While dedicated development tools manage most of my coding needs, I still regularly turn to AI language models for problem-solving, debugging, and understanding complex concepts. This combination of specialized coding tools and AI assistants has significantly improved my development speed and code quality. The future of coding is here, and it's powered by AI - not to replace developers, but to enhance our capabilities and let us focus on what matters most: solving real problems and building great software.

Vision Agent: Transforming Business Through AI-Powered Computer Vision

DonvitoCodes — Tue, 22 Oct 2024 07:44:21 GMT

Landing AI has launched Vision Agent, a powerful new tool that makes it simpler to create computer vision applications using Python. The company was founded by AI expert Andrew Ng, who also co-founded Google Brain, Coursera, and DeepLearning.AI, and continues to make AI technology more accessible to everyone.

What is Vision Agent?

Vision Agent is a development tool that helps programmers build computer vision applications quickly and easily. It stands out for its ability to detect and count objects, and automatically label images—features that are especially useful for automating visual tasks.

https://youtu.be/4HqjROcORuQ

Main Features

Simplified Development Process

Automated Workflows: Handles the entire process of building and launching vision applications
Smart Model Selection: Helps you choose the right AI model for your needs
Code Creation: Automatically writes Python code for your applications
Performance Tools: Makes sure your application runs efficiently

Smart Tools and Automation

Quick Testing: Build and test vision solutions faster
AI-Powered Labeling: Makes data labeling faster and easier
Automatic Processing: Creates code instantly for handling results
Object tracking and detection
Counting systems
Image labeling
Works with existing AI systems

Vision AI Technology's Potential Use Cases for Business

Vision Agent and similar AI technologies are transforming business operations across multiple sectors, offering powerful solutions to common challenges:

Manufacturing & Quality Control

Vision AI enhances production efficiency through:

Real-time defect detection during production
Automated quality inspection systems
Assembly line monitoring and verification
Reduction in inspection costs and human error
Predictive maintenance through visual monitoring

Healthcare & Medical Services

Medical facilities can improve patient care with:

Enhanced analysis of X-rays, MRIs, and CT scans
Early disease detection and diagnosis
Automated patient screening processes
Medical record digitization and management
Operating room workflow optimization

Retail & E-commerce

Businesses can optimize operations through:

Automated inventory tracking and management
Real-time shelf monitoring and restocking alerts
Customer behavior analysis and heat mapping
Self-checkout systems and theft prevention
Visual search capabilities for online shopping

Agriculture & Farming

Modern farming operations benefit from:

Drone-based crop monitoring and analysis
Early detection of plant diseases and pests
Automated yield estimation and forecasting
Precision farming and resource optimization
Harvest timing optimization

Automotive & Transportation

The transport sector can improve safety and efficiency with:

Advanced driver assistance systems (ADAS)
Autonomous navigation capabilities
Traffic monitoring and analysis
Vehicle damage inspection automation
Parking space management systems

Logistics & Warehouse Management

Distribution centers can streamline operations through:

Automated package sorting and routing
Inventory tracking and optimization
Storage space utilization analysis
Loading/unloading automation
Quality control during packaging

Construction & Real Estate

The building sector can enhance projects using:

Site safety monitoring and compliance
Progress tracking and documentation
Equipment utilization monitoring
Structural inspection automation
Virtual property tours and inspectionsBenefits for Businesses

Using Vision Agent brings several key advantages:

Less manual work needed for visual inspections
More accurate quality control
Lower operating costs
Better decision-making based on data
Faster processing of visual information

Benefits for Developers

Vision Agent makes developers' work easier through:

Faster Development: Build and test applications quickly
Less Coding Required: Automatic code generation saves time
Smoother Workflows: Automated processes reduce repetitive work
Better Data Handling: Makes labeling and processing data easier
Smart Resource Use: Picks the best tools for each task

What's Next

As tools like Vision Agent continue to improve, we expect to see:

More people using computer vision technology
More industries adopting these tools
New ways to use the technology
Better integration with current systems
Continued advances in AI capabilities

The Future of Vision AI

Vision Agent is an important step forward in making computer vision technology easier to use for both developers and businesses. As part of Landing AI's toolkit, it shows how committed the company is to making AI technology available to more people. With its practical features and uses across many industries, Vision Agent is set to play a key role in the future of automated visual processing.

The tool arrives at a time when businesses are looking for better ways to automate their visual inspection and analysis processes. By making complex computer vision technology easier to use, Vision Agent helps more businesses take advantage of advanced AI capabilities.

Selecting the Right AI Tool for your Business

DonvitoCodes — Tue, 22 Oct 2024 06:38:27 GMT

In today's rapidly evolving technological landscape, choosing the right AI tool has become a critical decision for organizations of all sizes. This framework provides a structured approach to evaluating, implementing, and optimizing AI tools for your specific needs.

1. Evaluation: Start with Hands-On Experience

The journey begins with firsthand experience. Before making any significant commitments:

Conduct a pilot program with a small test group
Document initial impressions and pain points
Set clear metrics for success
Compare the tool against existing solutions

💡

Pro Tip: Create a standardized evaluation checklist to ensure consistency across different tools and user groups.

2. Workflow and Use Case Fit

Alignment with existing workflows is crucial for successful adoption:

Map out current processes and identify integration points
Assess the learning curve for different user groups
Evaluate compatibility with existing tools and systems
Consider scalability for future needs

💡

Remember: The best AI tool is one that enhances rather than disrupts your existing workflows.

3. Feedback Collection and Assessment

Systematic feedback gathering provides valuable insights:

Implement structured feedback channels
Conduct regular user surveys
Track usage patterns and pain points
Create a feedback loop for continuous improvement

Key Areas for Feedback:

User experience and interface
Integration challenges
Performance metrics
Time savings and efficiency gains

4. Advantages and Disadvantages Analysis

Conduct a thorough analysis of pros and cons:

Advantages to Consider:

Performance improvements
Cost savings
Time efficiency
Quality enhancement
Innovation potential

Disadvantages to Watch For:

Implementation challenges
Training requirements
Technical limitations
Privacy concerns
Hidden costs

5. Budget Considerations

Financial planning must account for:

Initial purchase or subscription costs
Implementation expenses
Training and onboarding costs
Maintenance and updates
Potential ROI calculations

💡

Important: Factor in both direct and indirect costs for a complete financial picture.

6. Making the Decision

The decision-making process should be:

Data-driven
Transparent
Inclusive of key stakeholders
Aligned with organizational goals

Decision Criteria Checklist:

[ ] Meets core requirements
[ ] Falls within budget constraints
[ ] Shows clear ROI potential
[ ] Has stakeholder buy-in
[ ] Aligns with security policies

7. Implementation Strategy

A successful rollout requires:

Clear communication plan
Phased implementation approach
Dedicated support resources
Training programs
Success metrics tracking

Implementation Timeline Example:

Phase 1: Initial setup and configuration
Phase 2: Pilot group deployment
Phase 3: Department-wide rollout
Phase 4: Organization-wide implementation
Phase 5: Optimization and scaling

8. Productivity Enhancement

Maximize value through:

Regular training sessions
Best practices documentation
Use case repositories
Success story sharing
Continuous optimization

The Continuous Improvement Loop

Remember that tool selection is not a one-time decision. Establish a regular review cycle to:

Reassess tool performance
Evaluate new alternatives
Update requirements
Optimize processes
Consider emerging technologies

Selecting and implementing the right AI tool is a complex but manageable process when approached systematically. By following this framework, organizations can make informed decisions that drive real value and maintain competitive advantage in an AI-driven world.

Next Steps

Create your evaluation checklist
Identify key stakeholders
Set clear success metrics
Begin your pilot program
Document your journey

Remember: The goal is not just to implement AI, but to enhance your organization's capabilities in meaningful and measurable ways.

💡

What has been your experience with implementing AI tools in your organization? Share your insights and lessons below?

Self-Host NextJS in Fly.io: A Quick Try

DonvitoCodes — Sun, 13 Oct 2024 07:07:05 GMT

Recently, Lee Robinson of Vercel gave a tutorial on how to self-host NextJS in a VPS. He used DigitalOcean. I also tried it in Hetzner using Ubuntu instance and it worked.

The example used Docker to deploy NextJS. It has all the dependencies like Nginx and a Postgres database, also run in Docker. It used docker-compose to run all the docker containers required for the demo.

I read the code and I saw that the Dockerfile was pretty straightforward. So I thought since it is just running in docker, it should work in fly.io too, so I tried. I made a simpler example with only the NextJS app and tried to deploy it in Fly.io.

Fly.io is a platform where you can deploy your app easily. I was also able deploy an API route within NextJS which is called by a page as a client component. This is just a quick test for simple apps. I am not sure if all NextJS features can be supported. As for this test, I just needed the API routes to work to see if server-side rendering works as well.

https://www.youtube.com/watch?v=3PFsH14VxVU

The NextJS App

I created a new NextJS app using this command.

npx create-next-app@latest

Just answer the questions and it’ll create a fresh NextJS app for you.

Add a new page in /src/app/blog/page.tsx

'use client';

import { useState, useEffect } from 'react';
import Image from 'next/image';
import Link from 'next/link';

interface BlogPost {
  id: number;
  title: string;
  content: string;
  author: string;
  date: string;
  image: string;
}

export default function BlogPage() {
  const [blogPosts, setBlogPosts] = useState([]);

  useEffect(() => {
    async function fetchBlogPosts() {
      try {
        const response = await fetch('/api/blogs');
        if (!response.ok) {
          throw new Error('Failed to fetch blog posts');
        }
        const data = await response.json();
        setBlogPosts(data);
      } catch (error) {
        console.error('Error fetching blog posts:', error);
      }
    }

    fetchBlogPosts();
  }, []);

  return (
    "container mx-auto px-4 py-8">
      "text-3xl font-bold mb-8">Blog Posts
      "/" className="text-blue-500 hover:underline mb-4 inline-block">
        Back to Home
      
      {blogPosts.map((post) => (
        "mb-8 p-6 bg-white rounded-lg shadow-md">
          "flex flex-col md:flex-row">
            "md:w-1/3 mb-4 md:mb-0 md:mr-6">
              'https://picsum.photos/600'}
                alt={post.title}
                width={100}
                height={100}
                className="w-full h-auto rounded-lg"
              />
            
            "md:w-2/3">
              "text-2xl font-semibold mb-2">{post.title}
              "text-gray-600 mb-4">
                By {post.author} on {new Date(post.date).toLocaleDateString()}
              
              "text-gray-800 mb-4">{post.content.slice(0, 200)}...
              "https://donvitocodes.com" passHref>
                "inline-block bg-blue-500 text-white px-4 py-2 rounded hover:bg-blue-600 transition-colors duration-200">
                  Read More
                
              
            
          
        
      ))}
      "mt-8 text-center">
        "https://donvitocodes.com" target="_blank" rel="noopener noreferrer" className="text-blue-600 hover:underline">
          Visit donvitocodes.com
        
      
    
  );
}

Then, add an API route which was called within the blog page.

// saved in src/app/api/blogs/route.ts

import { NextResponse } from 'next/server';

export const revalidate = 0;

// Static data for blog posts
const blogPosts = [
  {
    id: 1,
    title: 'Getting Started with Next.js',
    content: 'Next.js is a powerful React framework that enables you to build server-side rendered and statically generated web applications using React. It provides a great developer experience with features like automatic code splitting, optimized performance, and easy deployment. Whether you\'re building a simple blog or a complex web application, Next.js offers the tools and flexibility to create fast, scalable, and SEO-friendly websites. Did you know that Next.js was created by Vercel and first released in 2016? Since then, it has become one of the most popular React frameworks, used by companies like Netflix, TikTok, and Twitch.',
    author: 'DonvitoCodes',
    date: '2023-05-01',
    image: 'https://picsum.photos/600'
  },
  {
    id: 2,
    title: 'The Power of TypeScript',
    content: 'TypeScript adds static typing to JavaScript, bringing a new level of robustness and maintainability to your code. With TypeScript, you can catch errors early in the development process, improve code readability, and enhance IDE support. It\'s particularly beneficial for large-scale applications, where it can significantly reduce bugs and improve team collaboration. Learn how TypeScript can supercharge your JavaScript development and make your code more reliable and easier to refactor. Fun fact: TypeScript was developed by Microsoft and is used in many of their projects, including Visual Studio Code. It\'s estimated that using TypeScript can reduce bug density by up to 15%!',
    author: 'DonvitoCodes',
    date: '2023-05-15',
    image: 'https://picsum.photos/600'
  },
  {
    id: 3,
    title: 'Building APIs with Next.js',
    content: 'Next.js provides an easy way to create API routes, allowing you to build full-stack applications with ease. With API routes, you can handle server-side logic, interact with databases, and create RESTful endpoints all within your Next.js project. This seamless integration between frontend and backend makes Next.js an excellent choice for developers looking to build modern web applications. Discover how to leverage Next.js API routes to create powerful, efficient, and scalable APIs for your projects. Did you know that Next.js API routes support serverless functions out of the box? This means you can deploy your APIs without managing server infrastructure, leading to cost savings and improved scalability.',
    author: 'DonvitoCodes',
    date: '2023-06-01',
    image: 'https://picsum.photos/600'
  },
  {
    id: 4,
    title: 'Deploying Next.js with Fly.io',
    content: 'Fly.io is a platform for deploying Next.js apps that offers a seamless experience for developers. With Fly.io, you can easily deploy your Next.js applications globally, ensuring low latency and high availability for users around the world. This platform provides features like automatic SSL, custom domains, and easy scaling options. Learn how to leverage Fly.io\'s powerful infrastructure to deploy your Next.js applications quickly and efficiently, and discover best practices for optimizing your app\'s performance in a production environment. Interestingly, Fly.io uses a unique approach called "edge computing" to run your applications closer to your users, potentially reducing latency by up to 50% compared to traditional cloud hosting.',
    author: 'DonvitoCodes',
    date: '2024-10-13',
    image: 'https://picsum.photos/600'
  },
  {
    id: 5,
    title: 'The Rise of JAMstack',
    content: 'JAMstack, which stands for JavaScript, APIs, and Markup, is revolutionizing web development. This modern architecture pattern focuses on decoupling the frontend from the backend, resulting in faster, more secure, and highly scalable websites. By leveraging static site generators, serverless functions, and content delivery networks, JAMstack sites can achieve incredible performance and developer productivity. Did you know that sites built with JAMstack can be up to 10 times faster than traditional dynamic websites? Explore how JAMstack can transform your web development process and deliver lightning-fast experiences to your users.',
    author: 'DonvitoCodes',
    date: '2024-11-01',
    image: 'https://picsum.photos/600'
  },
  {
    id: 6,
    title: 'Mastering CSS Grid Layout',
    content: 'CSS Grid Layout is a powerful tool for creating complex, responsive layouts with ease. It introduces a two-dimensional grid system to CSS, allowing for more flexible and intuitive design possibilities. With CSS Grid, you can create magazine-style layouts, complex dashboard interfaces, and responsive designs without relying on external frameworks. A fascinating tidbit: CSS Grid was in development for nearly 20 years before being widely adopted by modern browsers in 2017. Learn how to harness the full potential of CSS Grid and revolutionize your approach to web layout design.',
    author: 'DonvitoCodes',
    date: '2024-11-15',
    image: 'https://picsum.photos/600'
  }
];

export async function GET() {
  console.log('API /blogs called at ' + new Date().toISOString());
  return NextResponse.json(blogPosts);
}

You can also clone my repo and just change the files for your purpose

https://github.com/donvito/next-self-host-fly

Create a Dockerfile

Fly.io uses Docker to build and run your application. Here's an example Dockerfile for a Next.js app. I just copied this from Lee Robinson’s next-self-host demo.

FROM oven/bun:alpine AS base

# Stage 1: Install dependencies
FROM base AS deps
WORKDIR /app
COPY package.json bun.lockb ./
RUN bun install --frozen-lockfile

# Stage 2: Build the application
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN bun run build 

# Stage 3: Production server
FROM base AS runner
WORKDIR /app
ENV NODE_ENV=production
COPY --from=builder /app/public ./public
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static

EXPOSE 3000
CMD ["bun", "run", "server.js"]

Deploying to Fly.io

1. Install the Fly CLI

If you haven't already, install the Fly CLI using Homebrew if you’re using a Mac. If you are using Linux or Windows, you can follow their guide https://fly.io/docs/flyctl/install/

brew install flyctl

3. Initialize Your Fly.io App

Navigate to your NextJS project directory and run:

fly launch

This command will guide you through creating a new app on Fly.io and generate a fly.toml configuration file.

4. Create a fly.io launch configuration

The fly.toml file is automatically created when you run

fly launch

It will generate this fly.toml configuration similar to this. This is how a fly.toml file looks. You can adjust configuration based on your needs.

# fly.toml app configuration file generated for nextjs-fly-polished-resonance-4569 on 2024-10-13T11:00:00+08:00
#
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
#

app = 'nextjs-fly-polished-resonance-4569'
primary_region = 'syd'

[build]

[http_service]
  internal_port = 3000
  force_https = true
  auto_stop_machines = 'stop'
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']

[[vm]]
  memory = '1gb'
  cpu_kind = 'shared'
  cpus = 1

5. Run fly deploy for rebuilding and deploying your code

fly deploy

Issues

Although the app is working fine, I am getting this error for the production deployment. It seems sharp hasn’t been installed properly. I’m still trying to figure it out. sharp is used for NextJS optimisation and is a required dependency. If you know the solution, you can comment below.

syd [info]API /blogs called at 2024-10-13T04:48:31.159Z
syd [info] ⨯ Error: 'sharp' is required to be installed in standalone mode for the image optimization to function correctly. Read more at: https://nextjs.org/docs/messages/sharp-missing-in-production
syd [info]API /blogs called at 2024-10-13T04:48:37.506Z

That’s it!

This is just a quick test to deploy simple NextJS apps to fly.io if you want to self-host it. I didn’t test all NextJS features so I this is not for production use yet.

I am a Developer and This is My AI Story

DonvitoCodes — Tue, 08 Oct 2024 06:01:18 GMT

I've been working with AI - using it for productivity and developing apps with it. I started using AI when ChatGPT was released in November 2022. I was really amazed on how it can answer questions and produce code. I am someone who would try out a new technology and I continued using ChatGPT. I started with just asking questions about anything. It was fun and enjoyed it.

After a while, I realised that this thing -- ChatGPT was really smart and can be useful for helping out with some tasks as a developer. So I tried asking it to help out with code. During that time, its knowledge was not yet up to date and it didn't know the latest version of Go. But that didn't stop me from using it for coding because with its stock knowledge, it was able to help me with some trivial coding tasks. It also helped me understand concepts which I don't understand.

On June 21, 2022, Github Copilot was launched. Wow, I realised this was huge. I felt that this product was really a game-changer. Since Github Copilot is integrated with Visual Studio Code and Jetbrains IDEs as a plugin, using AI in coding would be seamless. You guessed it right, I tried it out. Since Github Copilot just had a free trial, I eventually paid for it. I used it for my coding and I got really good help from its auto-complete feature.

So this was the point when Github Copilot and ChatGPT were my go-to AI tools for coding.

After using Github Copilot for a few months, I realized that its auto-complete feature somewhat broke for me. I didn't really understand what happened. So I just used the chat feature that came with it. After a while, I realised I was not getting my money's worth with Github Copilot and just used ChatGPT for coding. So I just copy pasted code to and from ChatGPT. It was not convenient but it saved me some money from the Copilot subscription.

Github Copilot was exciting but it was not really a game-changer. Bummer!

I moved on and continued with ChatGPT.

Along the way, someone suggested to me to use Codeium. Codeium has the same exact functionality as Github Copilot. It works with Visual Studio Code, Jetbrains and even neovim! The free tier was really generous and I never hit the limit. I was happy!

Back at work, I was assigned to help our founder for a technical demo in Tech Week Singapore last year. That was around Oct 2023. Deadline was tight so I had to use AI to build it. Coding an AI product using AI was indeed an interesting use case. So the product was called HomerGPT, the idea was to build an AI chat application which is knowledgeable about Singapore real estate, our domain knowledge. Our tech stack was NextJS for the frontend and Go for our backend APIs. Luckily, I found a Vercel template which does exactly what we needed in the UI and backend part. The only catch is that the example was in TypeScript. Sheesh, I don't even know TypeScript!

I needed APIs for the backend and the examples OpenAI integration which I've tried out were in Python. Python is not even in our backend tech stack. Luckily, I found a great library, Vercel's AI SDK. So I decided to do the app in one language, TypeScript. I also deployed the APIs in NextJS, together with the frontend code which was not ideal. But, it was the best option due to the limited time.

After days of hard work, we were able to do the demo of HomerGPT in TechWeek Singapore successfully.

Job's not done yet! Remember I deployed the backend APIs in NextJS? I had to move that out. I thought of moving it to Go but thinking forward, it would not be a good decision. AI is mostly delivered with Python so I migrated the TypeScript code to Python. I asked ChatGPT to migrate the code from TypeScript to Python for me. Voila, it took me only a a week to migrate all the code from TypeScript to Python. Yes, that's from code migration to production.

I was impressed with the iterations and relationship I had with these AI tools. AI was a real assistant. AI even did the more difficult work, imagine that!

A few months passed, I left my job and started my learning journey. I left my previous role since I wanted to take a career break and accelerate my learning and experience with AI. Frontend was my weakest skill so I learned the basics of React and NextJS, a fullstack framework since I wanted to build AI apps. I learned NextJS to the point that I can start a project and deploy it to production. I thought that was good enough to go live with an app.

While I was learning and engaging with the community on X, previously known as Twitter, I came to know about very good tools to produce new user interfaces and components and even a new IDE.

I'm not a frontend developer! How can I make UI?!

Fret not, here are some AI tools I discovered.

Vercel v0

v0.dev and is developed by Vercel. These web based tools were designed to allow you to generate user interface components fast. Generated components uses Tailwind CSS and shadcn. v0 also has a, v0.dev/chat chat feature which allows developers to interate while generating the components. Adding the UI components to your project is also a breeze. It's just one command you need to execute and the UI component gets added to your project.

I created a website with just a single prompt using v0. Check it out.

https://www.youtube.com/watch?v=lQVKbEYNyT4

Claude AI

Claude AI also released their artifact feature with its most powerful model for coding Claude 3.5 Sonnet. This killed ChatGPT for me. I don't use ChatGPT anymore and even canceled my subscription. You can learn more about it here.

https://youtu.be/rHqk0ZGb6qo?si=TWA8u4YjpFJUNZJZ

Hold your breath!

When ChatGPT was launched, everyone was amazed. As a developer, I had the same feeling of excitement when I used Cursor the first time. I said to myself "Sh*t! this is the real game-changer!"

https://x.com/rickyrobinett/status/1825581674870055189

Cursor AI

Cursor has inline code editing which makes it really seamless to work with code. It's not perfect but most of the time gets the job done. It also has a Composer feature which allows you to generate multiple files. The user experience is great and the Cursor team just nailed it!

Check out this compilation of Cursor applications

https://x.com/ai_for_success/status/1843291106894577838

Walk the talk will you?!

Yes, of course!

Here's a list of apps I've built with AI.

AI Dream Photo - Unleash your dreams with AI Dream Photo! Transform into a successful businessman, an explorer on Mars, a tourist in Paris, or a rockstar on stage. Create stunning photos for your social media profiles, professional headshots, or just for fun to share with friends.
HomerGPT - HomerGPT is an AI-powered chat application designed specifically for the Singapore real estate market. It provides users with personalized property recommendations and insights, making the home-buying process more efficient and informed.
CondosPH - This is my website for my real estate side jobs. It is showcases properties for resale and a home loan calculator.
AI Prompt Creator - This tool assists users in creating and editing prompts for ChatGPT. It has more than 300 pre-defined prompts.
AI Prompt Library - This Chrome extension allows you to save and reuse prompts you frequently use in ChatGPT.
PDF to Images - This web application allows users to convert PDF documents into image formats quickly. It is particularly useful for extracting visual content from PDFs, making it easier to share and utilize in other projects.
Curated v0.dev Generations - I made this tool so we can easily discover and reuse really good v0.dev generations.
React Native Chat App using OpenAI - This is a React Native project starter for a mobile application built with Expo. It includes OpenAI integration.

So that's it. This is my AI journey. I hope you get to learn a few things out of it.

You can reach me at my website donvitocodes.com if you'd like me to share my story with your team.

Cheers and let's enjoy our AI journey!

How to Build Your AI App: A Comprehensive Startup Simulator Course

DonvitoCodes — Fri, 27 Sep 2024 04:59:43 GMT

Hi everyone!

Months back, I initially thought of doing a generic course to teach AI to non-technical people. I validated my idea and spoke to a few friends and previous colleagues. Based on my observation, people still have qualms on using AI. For the ordinary people I talked to, they don't see the usefulness of AI yet. I am also not able to convince them on how they can use it for their daily life or work. Some people simply don't have the time to try it out. So in short, I dropped the idea...

Motivation

On the brighter side, I have spoken to a some people in my discovery calls and realise that they are asking for ideas on how to build an app, learn about the process and technical side of things. They have entrepreneurial mindsets and they are looking to see how to earn from developing products, not necessarily AI apps.

So now I am drafting a course titled "How to Build Your AI App: A Comprehensive Startup Simulator Course". Basically, in this course, I'll be sharing what I learned for a few months in my indie hacking journey and of course my past experiences.

Target Audience

I aim to make the course useful for my target audience:

Developers who want to build stuff on their own or with a small team
Non-technical / business people who wants to understand the technical side of building an app
Students who are interested in building their own app for their startup idea

The course will have two(2) parts

Typical I talk, you listen - the boring part
Role Playing simulations (optional - only for those interested in having fun!) To apply what they learned, participants group into small teams and will take on roles to ideate/conceptualise an AI product and build a prototype using AI tools. The important part is to apply the concepts we learned during the course.

I also want to ask. Do you think people will be interested in part 2 - the simulation part?

Course Delivery

The course will be totally online so anyone from anywhere in the world can participate. I have a great tool to use for this type of collaboration which I can't share right now since it can be my competitive advantage.

The course is also a great networking opportunity since I plan to attract technical and non-technical people.

I attached my mind map of the outline below. Let me know if I need to add something you would like to know more about building an app.

That's all for now.

You can subscribe here so you can get updates about the course by email.

Cheers!

P.S. You can also share this to someone who might be interested.

OpenAI Structured Outputs: Why is it important?

DonvitoCodes — Thu, 15 Aug 2024 06:44:05 GMT

Building reliable software often means ensuring that the data we work with follows strict formats or "schemas". Before the recent OpenAI update about structured output support, it didn’t guarantee that the data would always fit the exact structure developers needed.

On August 6, 2024, OpenAI announced support for Structured Outputs in the API. This new feature is designed to ensure that the data generated by OpenAI models perfectly matches the JSON Schemas you provide. Structured Outputs make it easier for developers to create applications that require precise and structured data, whether it’s for building smart assistants, automating data entry, or creating complex workflows.

In OpenAI tests, the latest model, gpt-4o-2024-08-06, using Structured Outputs, achieved a perfect score in following complex JSON schemas, compared to less than 40% for the older gpt-4-0613 model. This shows just how much more reliable and accurate Structured Outputs are in the latest update.

Why is Structured Outputs important?

Consistency in AI-generated outputs is crucial when integrating with different systems. When building applications that rely on AI, having data that adheres to a specific structure ensures smoother and more reliable integration. Without consistent output, developers often need to add extra layers of validation and correction, which can complicate the process and introduce potential points of failure. Structured Outputs eliminate this uncertainty by guaranteeing that the data generated by the AI models fits the exact schema you’ve defined. This makes it easier to plug AI-generated data directly into your systems without worrying about mismatches or errors.

This consistency also leads to more stable integrations. When you can trust that the data from OpenAI will always follow the same structure, you can confidently build workflows that depend on that data. Whether it’s feeding information into a database, triggering actions in another application, or performing complex multi-step processes, Structured Outputs ensure that every piece of data will be just as you expect, reducing the need for additional processing and making your entire system more efficient and reliable.

In this post, we'll make a simple example of how to use Structured Outputs. Basically, here is a high level overview of how it works.

Study Plan Generator Example

In this example, we'll make a simple Study Plan Generator sends a request to the OpenAI model gpt-4o-2024-08-06 to create a study plan. The OpenAI model returns a consistent JSON format as we specified in the response_format parameter. We pass the pydantic model StudyPlan so OpenAI will know that it needs to adhere to this format when sending back the response.

completion = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    max_tokens=1024,
    temperature=0,
    messages=[
        {"role": "system", "content": "You are a helpful math tutor and is an expert in creating study plans for homeschooling kids."},
        {"role": "user", "content": "create a study plan to learn Algebra basics to my 14 year old"},
    ],
    response_format=StudyPlan,
)

Then the JSON data from the OpenAI response, containing all the structured information of the study plan, is then used to produce both a PDF document and an HTML file.

Key Components

Pydantic Models (Lesson, Module, StudyPlan):

The code defines several Pydantic models to structure the data:

  class Lesson(BaseModel):
      title: str
      description: str
      duration_minutes: int

  class Module(BaseModel):
      title: str
      description: str
      objectives: List[str]
      lessons: List[Lesson]    

  class StudyPlan(BaseModel):
      name: str
      start_date: str
      end_date: str
      notes: List[str]
      modules: List[Module]

These models ensure that the AI's output conforms to specific attributes, such as title, description, and lessons for each module in the study plan. This structuring is crucial for consistent and reliable data integration.

Interacting with OpenAI:

The code initializes an OpenAI client and sends a request to generate a study plan.

  client = OpenAI()

  completion = client.beta.chat.completions.parse(
      model="gpt-4o-2024-08-06",
      max_tokens=1024,
      temperature=0,
      messages=[
          {"role": "system", "content": "You are a helpful math tutor and is an expert in creating study plans for homeschooling kids."},
          {"role": "user", "content": "create a study plan to learn Algebra basics to my 14 year old"},
      ],
      response_format=StudyPlan,
  )

Here, the model is instructed to act as a math tutor and create a study plan for Algebra. The use of response_format=StudyPlan indicates that the model’s output should strictly follow the StudyPlan schema defined earlier.

Parsing and Validating the Response:

Once the AI generates the study plan, the response is parsed and validated against the StudyPlan schema:

  event = completion.choices[0].message.parsed

  # Convert the Pydantic model instance to JSON (without indent)
  json_string = event.json()

  # Validate the JSON string against the StudyPlan schema
  try:
      study_plan_dict = json.loads(json_string)
      study_plan = StudyPlan.model_validate(study_plan_dict)
      print("The JSON string is valid and adheres to the StudyPlan schema.")
  except Exception as e:
      print("Validation error:", e)

The StudyPlan.model_validate() function checks if the JSON string conforms to the expected schema, ensuring that the output is both valid and consistent.

Converting and Saving the JSON:

The validated study plan is then converted to a formatted JSON string and saved to a file:

  # Format JSON string with indentation using the built-in json module
  formatted_json = json.dumps(json.loads(json_string), indent=4)

  with open("study_plan.json", 'w') as json_file:
      json_file.write(formatted_json)

  print(formatted_json)

This step ensures that the AI-generated study plan is saved in a human-readable and structured format, ready for further use or sharing.

Generating HTML and PDF Outputs:
- Finally, the code converts the JSON data into HTML and PDF formats:
```
  generate_html(formatted_json, "study_plan.html")
  generate_pdf(formatted_json, "study_plan.pdf")
```
- These functions likely take the structured JSON and format it into visually appealing documents, enhancing the accessibility and utility of the study plan.

Full Source Code

main.py

from pydantic import BaseModel
from typing import List, Optional
from openai import OpenAI
import json

from pydantic import BaseModel
from typing import List, Optional

from output_generators import generate_html, generate_pdf

class Lesson(BaseModel):
    title: str
    description: str
    duration_minutes: int

class Module(BaseModel):
    title: str
    description: str
    objectives: List[str]
    lessons: List[Lesson]    

class StudyPlan(BaseModel):
    name: str
    start_date: str
    end_date: str
    notes: List[str]
    modules: List[Module]

client = OpenAI()

completion = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    max_tokens=1024,
    temperature=0,
    messages=[
        {"role": "system", "content": "You are a helpful math tutor and is an expert in creating study plans for homeschooling kids."},
        {"role": "user", "content": "create a study plan to learn Algebra basics to my 14 year old"},
    ],
    response_format=StudyPlan,
)

event = completion.choices[0].message.parsed

# Convert the Pydantic model instance to JSON (without indent)
json_string = event.json()

# Validate the JSON string against the StudyPlan schema
try:
    study_plan_dict = json.loads(json_string)
    study_plan = StudyPlan.model_validate(study_plan_dict)
    print("The JSON string is valid and adheres to the StudyPlan schema.")
except Exception as e:
    print("Validation error:", e)


# Format JSON string with indentation using the built-in json module
formatted_json = json.dumps(json.loads(json_string), indent=4)

with open("study_plan.json", 'w') as json_file:
    json_file.write(formatted_json)

print(formatted_json)

generate_html(formatted_json, "study_plan.html")
generate_pdf(formatted_json, "study_plan.pdf")

output_generators.py

from fpdf import FPDF
from typing import Dict
import json
from pathlib import Path

def generate_html(json_data: str, output_file: str) -> None:
    # Parse JSON data
    study_plan = json.loads(json_data)

    # Start building the HTML content with CSS styling
    html_content = f"""
    
    
        <span class="hljs-subst">{study_plan[<span class="hljs-string">'name'</span>]}</span>
        
    
    
        
            Study Plan: {study_plan['name']}
            Start Date: {study_plan['start_date']}
            End Date: {study_plan['end_date']}
            Notes
            
    """

    for note in study_plan['notes']:
        html_content += f"{note}
"

    html_content += "
Modules"

    for module in study_plan['modules']:
        html_content += f"""
                    
            {module['title']}
            {module['description']}
            Objectives
            
        """
        for objective in module['objectives']:
            html_content += f"{objective}
"
        html_content += "
 Lessons
"

        for lesson in module['lessons']:
            html_content += f"""
            
                {lesson['title']}
                
                {lesson['description']}

                Duration:
                {lesson['duration_minutes']} minutes
            
            """

        html_content += "
"

    html_content += """
        
    
    
    """

    # Write HTML content to the specified output file
    with open(output_file, 'w') as file:
        file.write(html_content)

def generate_pdf(json_data: str, output_file: str) -> None:
    # Parse JSON data
    study_plan = json.loads(json_data)

    # Create PDF document
    pdf = FPDF()
    pdf.set_auto_page_break(auto=True, margin=15)
    pdf.add_page()

    # Add title
    pdf.set_font("Arial", 'B', 16)
    pdf.cell(0, 10, f"Study Plan: {study_plan['name']}", 0, 1, 'C')

    # Add study plan details
    pdf.set_font("Arial", '', 12)
    pdf.cell(0, 10, f"Start Date: {study_plan['start_date']}", 0, 1)
    pdf.cell(0, 10, f"End Date: {study_plan['end_date']}", 0, 1)

    # Add notes
    pdf.set_font("Arial", 'B', 14)
    pdf.cell(0, 10, "Notes:", 0, 1)
    pdf.set_font("Arial", '', 12)
    for note in study_plan['notes']:
        pdf.multi_cell(0, 10, f"- {note}")

    # Add modules with visual differentiation
    pdf.set_font("Arial", 'B', 14)
    pdf.cell(0, 10, "Modules:", 0, 1)

    for module in study_plan['modules']:
        pdf.set_font("Arial", 'B', 12)
        pdf.set_fill_color(240, 240, 240)  # Light gray background for module title
        pdf.cell(0, 10, f"Module Title: {module['title']}", 0, 1, 'L', fill=True)

        pdf.set_font("Arial", '', 12)
        pdf.multi_cell(0, 10, f"Description: {module['description']}")

        pdf.set_font("Arial", 'I', 12)
        pdf.cell(0, 10, "Objectives:", 0, 1)
        for objective in module['objectives']:
            pdf.multi_cell(0, 10, f"- {objective}")

        pdf.set_font("Arial", '', 12)
        pdf.cell(0, 10, "Lessons:", 0, 1)

        for lesson in module['lessons']:
            pdf.set_fill_color(255, 240, 240)  # Slightly colored background for lessons
            pdf.cell(0, 10, f"  Lesson Title: {lesson['title']}", 0, 1, 'L', fill=True)
            pdf.multi_cell(0, 10, f"  Description: {lesson['description']}")
            pdf.cell(0, 10, f"  Duration: {lesson['duration_minutes']} minutes", 0, 1)
            pdf.ln(5)

        pdf.ln(10)

    # Save the PDF to the specified output file
    pdf.output(output_file)

requirements.txt

annotated-types==0.7.0
anyio==4.4.0
certifi==2024.7.4
distro==1.9.0
fpdf==1.7.2
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
idna==3.7
jiter==0.5.0
openai==1.40.6
pydantic==2.8.2
pydantic_core==2.20.1
sniffio==1.3.1
tqdm==4.66.5
typing_extensions==4.12.2

💡

Source code also downloadable in my gist https://gist.github.com/donvito/d3927c09029770592b5644f04942b811

Running the example

Installing the dependencies using pip install then run the application.

pip install -r requirements.txt

python main.py

With Structured Outputs, we can now confidently build and integrate AI-driven solutions that are both precise and dependable, saving time and reducing the need for extensive validation or error handling. This new feature of OpenAI will make integration with different systems seamless and reliable.

Vercel AI SDK 3.3: Powering Up AI Development

DonvitoCodes — Wed, 07 Aug 2024 05:09:14 GMT

Vercel has just announced the release of Vercel AI SDK 3.3, bringing a host of new features and improvements to their toolkit for building AI applications with JavaScript and TypeScript. This update introduces exciting capabilities that will enhance developers' ability to create sophisticated AI-powered applications.

Key Updates

1. Tracing (Experimental)

Vercel AI SDK now supports OpenTelemetry for tracing, offering developers crucial insights into their AI applications. This feature allows for better observability, helping developers understand timing, token usage, prompts, and response content for individual model calls.

The new version introduces support for sending file attachments along with messages in AI chat applications. This feature enhances the useChat React hook, allowing developers to easily implement functionalities like image uploads in their chatbots.

3. useObject Hook (Experimental)

This new React hook enables streaming of structured object generation directly to the client. It's particularly useful for creating dynamic interfaces that display JSON objects as they're being streamed, opening up possibilities for more interactive and responsive AI applications.

4. Additional LLM Settings

Vercel has expanded the toolkit's language model capabilities with new features:

JSON schema support for tools and structured object generation
Stop sequences for more control over text generation
Custom headers support for enhanced flexibility in API calls

What This Means for Developers

Improved Observability: The tracing feature will make it easier to debug and optimize AI applications, especially when dealing with the non-deterministic nature of language models.
Enhanced User Experiences: Multi-modal attachments and the useObject hook will allow developers to create more sophisticated and interactive AI interfaces.
Greater Flexibility: The additional LLM settings provide more control and customization options when working with language models.

Vercel AI SDK represents a significant step forward in making AI development more accessible and powerful for JavaScript and TypeScript developers. Whether you're building chatbots, content generation tools, or complex AI-powered applications, these new features provide the tools needed to create more robust and interactive experiences.

As AI continues to play an increasingly important role in web development, tools like Vercel AI SDK are becoming indispensable for developers looking to stay at the forefront of technology. With this release, Vercel continues to demonstrate its commitment to empowering developers in the AI space.

Using Groq's Whisper API and Go for Transcribing Audio to Text

DonvitoCodes — Mon, 29 Jul 2024 03:44:44 GMT

In an age where communication happens through various mediums, the ability to convert spoken language into written text has become increasingly valuable. Speech-to-text technology has a wide range of applications that can enhance productivity, accessibility, and user experience.

Speech to Text Use Cases

Here are some examples of how you might use Speech To Text

Meeting Transcription: Automatically create text records of business meetings, conferences, or interviews.
Closed Captioning: Generate captions for videos, making content accessible to deaf or hard-of-hearing viewers.
Voice Commands: Enable voice control in apps or smart home devices by converting spoken commands to text.
Podcast Transcription: Create text versions of podcasts to improve SEO and make content searchable.
Voice Notes: Allow users to dictate notes or memos, which are then converted to text for easy editing and sharing.
Call Center Analytics: Transcribe customer service calls to analyze common issues, sentiment, and agent performance.
Medical Dictation: Help healthcare professionals create patient notes or reports by speaking rather than typing.
Legal Documentation: Transcribe court proceedings, depositions, or client interviews for accurate record-keeping.
Educational Content: Convert lectures or educational videos into text for students to review or for creating study materials.
Accessibility Tools: Help people with disabilities interact with digital content by converting audio to text.

These are just a few examples of how speech-to-text technology can be applied across various industries and scenarios. Now, let's dive into how Groq's Whisper API can help you implement this powerful technology in your own applications

What can Whisper API Do?

Groq's Whisper API can turn audio into text.

How to Use It

To use the API, you'll need to call use this API:

For transcription: https://api.groq.com/openai/v1/audio/transcriptions

The API uses a model called whisper-large-v3. This model is very good at transcribing audio.

Things to Remember

You can only upload files up to 25 MB in size.
The API can work with these types of audio files: mp3, mp4, mpeg, mpga, m4a, wav, and webm.
If your file has more than one audio track (like a video with different language tracks), the API will only use the first track.

Preparing Your Audio File

Before using the API, you might need to change your audio file a bit. The API works best with audio that has a certain quality (16,000 Hz mono). Here's how you can change your file using a tool called ffmpeg:

Copyffmpeg \
  -i  \
  -ar 16000 \
  -ac 1 \
  -map 0:a: \

Replace with the name of your audio file, and with what you want to call the new file.

Sample Code

package main

import (
    "bytes"
    "fmt"
    "io"
    "mime/multipart"
    "net/http"
    "os"
)

const (
    apiBaseUrl = "https://api.groq.com/openai"
    STTWhisperLargeV3 = "whisper-large-v3"
)

type GroqClient struct {
    ApiKey string
}

type GroqMessage struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

func main() {

    apiKey := os.Getenv("GROQ_API_KEY")
    gclient := &GroqClient{
        ApiKey: apiKey,
    }

    transcriptText, err := gclient.TranscribeAudio("audio.m4a")
    if err != nil{
        panic(err)
    }
    fmt.Println("transcriptText", transcriptText)


    fmt.Println("Transcript saved to transcript.txt")
}

func (g *GroqClient) TranscribeAudio(audioFile string) (string, error) {

    // File to upload
    filepath := audioFile
    file, err := os.Open(filepath)
    if err != nil {
        fmt.Println("Error opening file:", err)
        return "", err
    }
    defer file.Close()

    transcriptionUrl := "/v1/audio/transcriptions"
    finalUrl := fmt.Sprintf("%s%s", apiBaseUrl, transcriptionUrl)

    // Prepare form data
    body := &bytes.Buffer{}
    writer := multipart.NewWriter(body)

    // Add file field
    part, err := writer.CreateFormFile("file", filepath)
    if err != nil {
        fmt.Println("Error creating form file:", err)
        return "", err
    }
    _, err = io.Copy(part, file)
    if err != nil {
        fmt.Println("Error copying file:", err)
        return "", err
    }

    temp := "0"
    responseFormat := "json"
    language := "en"

    // Add other fields
    _ = writer.WriteField("model", STTWhisperLargeV3)
    _ = writer.WriteField("temperature", temp)
    _ = writer.WriteField("response_format", responseFormat)
    _ = writer.WriteField("language", language)

    // Close writer
    err = writer.Close()
    if err != nil {
        fmt.Println("Error closing writer:", err)
        return "", err
    }
    // Create POST request
    req, err := http.NewRequest(http.MethodPost, finalUrl, body)
    if err != nil {
        return "", err
    }

    // Set headers
    req.Header.Set("Content-Type", writer.FormDataContentType())
    req.Header.Set("Authorization", fmt.Sprintf("bearer %s", g.ApiKey)) // Replace with your actual access token

    // Create HTTP client
    client := &http.Client{}

    // Send request
    resp, err := client.Do(req)
    if err != nil {
        fmt.Println("Error sending request:", err)
        return "", err
    }
    defer func(Body io.ReadCloser) {
        err1 := Body.Close()
        if err1 != nil {

        }
    }(resp.Body)

    // Read response body
    responseBody, err := io.ReadAll(resp.Body)
    if err != nil {
        fmt.Println("Error reading response body:", err)
        return "", err
    }

    r := string(responseBody)

    return r, nil

}

With Groq's Whisper API, turning speech into text is now easier than ever. Whether you're building an app for meeting transcriptions, creating accessibility tools, or any of the other use cases we've explored, this technology can help you achieve your goals.

Give it a try in your next project and see how it can transform your approach to handling audio content!

If you're interested in learning more about developing apps with Generative AI, subscribe to my blog for more tutorials and sample code. You can also follow me in Twitter or connect with me in LinkedIn.

Integrate AI into Java Apps — No Python Required

DonvitoCodes — Fri, 19 Jul 2024 09:22:46 GMT

In this blog post, we'll explore how to leverage Spring AI to create a simple chat API using OpenAI's language model and Java. Before we go into the example, let's see why this is exciting!

Java Developers, Rejoice!

No need to learn a new language!

As a seasoned Java developer, you might think that diving into AI development requires learning a new language, such as Python. After all, Python is widely recognized for its robust AI libraries and frameworks. However, with Spring AI, Java developers can achieve AI integration without stepping outside their comfort zone.

Most corporate applications are built using Java due to its stability, scalability, and enterprise-level support. Integrating AI directly into these applications using Spring AI allows businesses to enhance their functionality and user experience without needing to overhaul their technology stack. It also ensures that developers can work within a familiar environment, improving productivity and reducing the learning curve.

Staying within the Java ecosystem means you can continue to use your existing skills and knowledge. You won't have to invest time and resources in learning a new language or adapting to different paradigms. This not only saves time but also helps maintain consistency in your codebase, making future updates and maintenance more manageable.

What is Spring AI?

Spring AI is a library that aims to make it easier for developers to create applications with artificial intelligence features. The main goal of Spring AI is to help connect your company's data and systems with AI models in a straightforward way.

💡

At its core, Spring AI addresses the fundamental challenge of AI integration: Connecting your enterprise Data and APIs with the AI Models.

The library offers a set of tools and features that developers can use as building blocks for AI applications. These include support for various AI model providers like OpenAI, Microsoft, Amazon, Google, and Hugging Face. It supports different types of AI models such as chat and image generation. Spring AI is designed to be flexible, allowing developers to easily switch between different components without having to rewrite a lot of code.

Spring AI Architecture

Here is the architecture of Spring AI*( source: official documentation)*

Supported models

OpenAI, Microsoft, Amazon, Google, Amazon Bedrock, Hugging Face and more.

Some key features of Spring AI include the ability to use AI models from different providers with a consistent interface, support for turning AI outputs into Java objects, and tools for managing and processing data for AI use. It also provides ready-to-use components that work well with Spring Boot, a popular framework for building Java applications. With these features, developers can more easily create applications that do things like answer questions based on company documents or have conversations using a company's knowledge base.

Seamless Java Integration

Spring AI is designed to integrate AI capabilities into Java applications effortlessly. It provides a straightforward way to connect with language models using Java, making it an excellent choice for those who want to keep their tech stack consistent. This library allows you to leverage the power of conversational AI without needing to rewrite your existing codebase in a different language.

Imagine adding a conversational interface to your enterprise application that can handle customer inquiries, provide personalized recommendations, or automate routine tasks. With Spring AI, these features can be implemented quickly and efficiently. You can focus on developing the business logic and user experience without worrying about the intricacies of a new programming language.

Prerequisites

Before we begin, make sure you have the following:

Java Development Kit (JDK) 17 or later
Maven
An OpenAI API key

Project Setup

Let's start by setting up a new Spring Boot project with the necessary dependencies.

pom.xml

First, we'll configure our pom.xml file:


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0modelVersion>
    <parent>
        <groupId>org.springframework.bootgroupId>
        <artifactId>spring-boot-starter-parentartifactId>
        <version>3.3.0version>
        <relativePath/> 
    parent>
    <groupId>com.donvitocodesgroupId>
    <artifactId>openai-javaartifactId>
    <version>0.0.1-SNAPSHOTversion>
    <name>OpenAI Javaname>
    <description>Simple AI Application using OpenAPI Servicedescription>
    <properties>
        <java.version>17java.version>
    properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.aigroupId>
                <artifactId>spring-ai-bomartifactId>
                <version>1.0.0-SNAPSHOTversion>
                <type>pomtype>
                <scope>importscope>
            dependency>
        dependencies>
    dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-webartifactId>
        dependency>

        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-actuatorartifactId>
        dependency>

        <dependency>
            <groupId>org.springframework.aigroupId>
            <artifactId>spring-ai-openai-spring-boot-starterartifactId>
        dependency>

        <dependency>
            <groupId>org.springframework.bootgroupId>
            <artifactId>spring-boot-starter-testartifactId>
            <scope>testscope>
        dependency>
    dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.bootgroupId>
                <artifactId>spring-boot-maven-pluginartifactId>
            plugin>
        plugins>
    build>

    <repositories>
        <repository>
            <id>spring-milestonesid>
            <name>Spring Milestonesname>
            <url>https://repo.spring.io/milestoneurl>
            <snapshots>
                <enabled>falseenabled>
            snapshots>
        repository>
        <repository>
            <id>spring-snapshotsid>
            <name>Spring Snapshotsname>
            <url>https://repo.spring.io/snapshoturl>
            <releases>
                <enabled>falseenabled>
            releases>
        repository>
    repositories>
project>

This configuration sets up our project with Spring Boot, Spring AI, and the necessary dependencies for working with OpenAI.

Application Structure

Now, let's create the main application class Application.Java

package com.donvitocodes;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

Configuration Class

Next, we'll create a configuration class Config.java to set up our ChatClient.

package com.donvitocodes;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.openai.OpenAiChatOptions;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
class Config {

    @Bean
    ChatClient chatClient(ChatClient.Builder builder) {

        String modelName = "gpt-4o-mini";
        float temperature = 0f;
        int maxTokens = 1024;

        return builder
                .defaultOptions(OpenAiChatOptions.builder()
                        .withModel(modelName)
                        .withTemperature(temperature)
                        .withMaxTokens(maxTokens)
                        .withStreamUsage(true)
                        .build())
                .build();
    }

}

This configuration sets up the ChatClient with custom options for the OpenAI model, including the model name, temperature, and maximum tokens.

Creating the Controller

Finally, let's create a controller:

package com.donvitocodes;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Flux;

import java.util.Map;

@RestController
class AIController {
    private final ChatClient chatClient;

    AIController(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    @GetMapping("/chat")
    Map completion(@RequestParam(value = "message", defaultValue = "") String message) {
        return Map.of(
                "completion",
                chatClient.prompt()
                        .user(message)
                        .call()
                        .content());
    }

    // Streaming 
    @GetMapping(value = "/chat-stream", produces = "application/stream+json")
    public Flux streamCompletion(@RequestParam(value = "message", defaultValue = "") String message) {
        return  chatClient.prompt()
                .user(message)
                .stream()
                .content();
    }


}

This controller exposes a simple GET endpoint that accepts a message parameter and returns the AI-generated response. The response is not streaming in this case.

Streaming the Chat Response

If you want to stream the response, this is that part of the code

@GetMapping(value = "/chat-stream", produces = "application/stream+json")
public Flux streamCompletion(@RequestParam(value = "message", defaultValue = "") String message) {
    return chatClient.prompt()
          .user(message)
          .stream()
          .content();
}

Running the Application

To run the application, make sure you've set your OpenAI API key as an environment variable:

export SPRING_AI_OPENAI_API_KEY=your_api_key_here

Then, you can start the application using Maven:

mvn spring-boot:run

Testing the API

URL:http://localhost:8080/chat

Method:GET

Query Parameter:

message (string): The prompt or instruction to be processed by the API. For example, "create a short blog post about investing in real estate".

Once your application is running, you can test the API using curl or any HTTP client:

curl --location 'http://localhost:8080/chat?message=create%20a%20short%20blog%20post%20about%20investing%20in%20real%20estate'

Or use Postman

Testing with Streaming

You need to use --no-buffer or -N when doing the curl. Make sure you call the http://localhost:8080//chat-stream endpoint.

curl --no-buffer --location 'http://localhost:8080/chat-stream?message=create%20a%20short%20blog%20post%20about%20investing%20in%20real%20estate%20in%20the%20philippines'

The API will return the AI-generated response based on the input message.

In this blog post, we've explored how to create a chat API using Spring AI, OpenAI, and Java. This integration provides a powerful foundation for building AI-powered conversational applications within the Spring ecosystem. By leveraging Spring AI's abstraction layer, you can easily switch between different AI providers or models in the future, making your application more flexible and maintainable.

Remember to handle API rate limits, implement proper error handling, and consider adding authentication to your API before deploying it to production.

Happy coding!

If you're interested in learning more about developing apps with Generative AI, subscribe to my blog for more tutorials and sample code. You can also follow me in Twitter or connect with me in LinkedIn.

No JavaScript, No Problem! Building Web Apps with HTMX and Go

DonvitoCodes — Fri, 05 Jul 2024 08:17:26 GMT

In this tutorial, we'll dive into using HTMX with Go to create a simple dynamic web application without the need for JavaScript. Our example will focus on a translation service application, demonstrating how to leverage HTMX to make API calls to a Go backend application.

https://www.youtube.com/watch?v=d3DrSoz-5no

What is HTMX?

HTMX is a modern JavaScript library that extends HTML through attributes, allowing you to define behaviour directly in your html. This can simplify the process of adding interactivity to your web pages, making it an excellent tool for developers looking to enhance their applications with minimal fuss.

Here's a diagram generated by Claude AI to show how HTMX works.

While HTMX provides significant benefits in terms of performance, rapid prototyping, and enhanced user experience, it may not be suitable for all projects. Complex, large-scale applications might still benefit from more comprehensive frameworks. However, for many web applications that prioritize simplicity, performance, and rapid development, HTMX offers a great alternative to traditional JavaScript-heavy implementations.

How to use HTMX

In this example, we have 3 main files index.html, translate.html and main.go

index.html

This file contains the clients side code which has a form which submits to a /translate API endpoint.

"en">
    
        HTMX + Go Example
    
    
        
        "container">
            HTMX + Go
            This simple example calls an LLM using Groq Completion API for the translation.
        "/translate" hx-target="#translated-text" hx-swap="replace">
            for="text-to-translate">Enter text to translate

            "text-to-translate"</span> name=<span class="hljs-string">"textToTranslate"</span> rows=<span class="hljs-string">"10"</span> cols=<span class="hljs-string">"50"</span>>
            


            for="translate-form">
            


            
        
        "translated-text">

HTMX Integration

This line imports the HTMX library from a CDN. HTMX is a library that allows you to access AJAX, CSS Transitions, WebSockets, and Server Sent Events directly in HTML, without writing JavaScript.

The Form

"/translate" hx-target="#translated-text" hx-swap="replace">

This form uses HTMX attributes:

hx-post="/translate": When submitted, it will make a POST request to the "/translate" endpoint.
hx-target="#translated-text": The response will be inserted into the element with id "translated-text".
hx-swap="replace": The response will replace the content of the target element.

main.go

func main() {

    // Set up the http handlers
    http.HandleFunc("/", handleHome)
    http.HandleFunc("/translate", handleTranslate)

    // Start the server
    err := http.ListenAndServe(":8080", nil)
    if err != nil {
       return
    }

}

This handleHome() function will serve index.html. This is the entrypoint of our web server which renders the html form with htmx enabled. This code is ready if you want to load data into htmx. I'll show that in a different example so we don't complicate it right now. For this example, we just want to show how we can submit a form and update the UI dynamically using the response of the API.


func handleHome(w http.ResponseWriter, r *http.Request) {
    tmpl := template.Must(template.ParseFiles("index.html"))

    err := tmpl.Execute(w, nil)
    if err != nil {
        return
    }
}

The handleTranslate() function handles the form submission. It writes translation.html into the response.

func handleTranslate(w http.ResponseWriter, r *http.Request) {
    textToTranslate := r.FormValue("textToTranslate")
    languageToTranslateTo := r.FormValue("languageToTranslateTo")
    translatedText := translateText(textToTranslate, languageToTranslateTo)

    type Translation struct {
        TextToTranslate string
        TranslatedText  string
    }

    t := Translation{
        TextToTranslate: textToTranslate,
        TranslatedText:  translatedText,
    }

    tmpl := template.Must(template.ParseFiles("translation.html"))
    err := tmpl.Execute(w, t)
    if err != nil {
        return
    }

}

Before sending the response back, it adds the translated text into the template.

t := Translation{
    TextToTranslate: textToTranslate,
    TranslatedText:  translatedText,
}
tmpl := template.Must(template.ParseFiles("translation.html"))
err := tmpl.Execute(w, t)

translation.html looks like this

"translated-text" class="translated-text">
    {{.TranslatedText}}

So what is happening?

The form submission calls the endpoint /translate
The Go API processes the request and retrieves the form values textToTranslate and languageToTranslateTo
After retrieving it, the flow calls the translation function using the Groq client to translate the text from the user.
It gets the response and populates the Translation struct named t.
Then, it populates the translation.html template using the struct. Basically tmpl.Execute() will get the values from the struct and and update the template. tmpl.Execute() will also save the response to w http.ResponseWriter
The response will then be received by the client and processed by HTMX. HTMX will update the UI.
- hx-target="#translated-text": The response will be inserted into the element with id "translated-text".
- hx-swap="replace": The response will replace the content of the target element.
  
  Here's the reference to the list of supported HTMX attributes you can use

💡

https://htmx.org/reference/

All of these actions, which would traditionally require custom JavaScript, are accomplished using just HTML attributes. This approach can significantly simplify your front-end code, reduce the amount of JavaScript you need to write and maintain, and make your application more accessible and easier to understand.

You can go through the entire example in my Github repo. Don't forget to leave a ⭐

https://github.com/donvito/learngo/tree/master/htmx

If you're interested in learning more about developing with Go, subscribe to my blog for more tutorials and sample code. You can also follow me in Twitter.

Chat with Data from a Website using LangChain and OpenAI

DonvitoCodes — Wed, 03 Jul 2024 12:53:42 GMT

In this tutorial, we will walk through the steps to create a simple Q&A system that retrieves and answers questions based on information from a specific Wikipedia page. This system leverages the power of LangChain, OpenAI's GPT-3.5-turbo, and LangChain Hub. By the end of this guide, you'll have a working example of how to integrate these tools to create a practical application.

Prerequisites

Before we start, ensure you have the following installed:

Python 3.11 or higher
pip (Python package installer)

You will also need to install the necessary dependencies. Open your terminal and run the following command:

pip install langchain openai langchain-community chromadb tiktoken langchainhub

Additionally, you need to set the OPENAI_API_KEY environment variable to authenticate with the OpenAI API. You can do this by running the following command in your terminal:

export OPENAI_API_KEY='your-openai-api-key'

Replace 'your-openai-api-key' with your actual OpenAI API key.

Step 1: Setting Up the Project

First, let's import the required modules and initialize the document loader.

from langchain import hub
from langchain.chains import RetrievalQA
from langchain_community.chat_models import ChatOpenAI
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.embeddings import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_text_splitters import RecursiveCharacterTextSplitter

Step 2: Loading and Splitting the Document

We will load a Wikipedia page using WebBaseLoader. For this tutorial, we'll use the Wikipedia page of Korean actress Kim Ji-won. After loading the data, we will split the document into smaller chunks to make it easier for our model to process.

# You can add multiple URLs here
loader = WebBaseLoader(["https://en.wikipedia.org/wiki/Kim_Ji-won_(actress)"])
data = loader.load()

text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
all_splits = text_splitter.split_documents(data)

Step 3: Creating the Vector Store

Next, we need to convert the text chunks into embeddings using OpenAI's embeddings model and store these embeddings in a vector store. We'll use Chroma as our vector store.

vectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())

Step 4: Setting Up the Prompt

We'll use a pre-defined prompt from LangChain Hub. Prompts are essential as they guide the model on how to structure its responses.

prompt = hub.pull("donvito-codes/rag-prompt")

Here is the Prompt configured in LangChain Hub

Step 5: Initializing the Language Model

We will use OpenAI's GPT-3.5-turbo model for our Q&A system. The temperature parameter controls the randomness of the model's responses. A lower temperature means more deterministic outputs.

llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)

Step 6: Creating the RetrievalQA Chain

We will create a RetrievalQA chain using the language model and the vector store retriever. The chain_type_kwargs parameter allows us to pass additional settings to the chain, such as the prompt.

qa_chain = RetrievalQA.from_chain_type(
    llm,
    retriever=vectorstore.as_retriever(),
    chain_type_kwargs={"prompt": prompt}
)

Step 7: Asking a Question

Now, we can ask a question and get an answer from our Q&A system. For this example, we'll ask about Kim Ji-won's latest TV series.

question = "Which latest TV series did Kim Ji-won star in 2024?"
result = qa_chain({"query": question})
print(result["result"])

Here's the full working source code

from langchain import hub
from langchain.chains import RetrievalQA
from langchain_community.chat_models import ChatOpenAI
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.embeddings import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_text_splitters import RecursiveCharacterTextSplitter

loader = WebBaseLoader(["https://en.wikipedia.org/wiki/Kim_Ji-won_(actress)"])
data = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
all_splits = text_splitter.split_documents(data)

vectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())

prompt = hub.pull("donvito-codes/rag-prompt")

llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)

# RetrievalQA
qa_chain = RetrievalQA.from_chain_type(
    llm,
    retriever=vectorstore.as_retriever(),
    chain_type_kwargs={"prompt": prompt}
)

question = "which latest tv series did Kim Ji Won star on?"
result = qa_chain({"query": question})
print(result["result"])

Save the file as main.py and run it

python main.py

You've now created a basic Q&A system that pulls data from a Wikipedia page, processes it into embeddings, and uses a language model to retrieve and answer questions. This is a powerful example of how you can leverage LangChain and OpenAI to create intelligent applications.

Feel free to experiment with different Wikipedia pages, questions, and prompts to see how the system performs. The flexibility of LangChain and OpenAI allows for a wide range of applications beyond just Q&A systems. Happy coding!

Additional Resources

If you're interested in learning more about developing with Generative AI, subscribe to my blog for more tutorials and sample code.

LaunchStack: My NextJS Learning Journey

DonvitoCodes — Wed, 03 Jul 2024 05:12:14 GMT

Hey there, fellow software engineers and aspiring startup founders! I'm excited to share a project I'm just starting: LaunchStack. But here's the twist – I'm not an expert. In fact, I'm using this project to learn NextJS myself!

What is LaunchStack?

LaunchStack is going to be a NextJS starter kit that I hope will eventually have everything needed to build a startup or website with dynamic features quickly. But right now, it's more of a dream and a learning goal than a finished product.

Why am I creating LaunchStack?

Learning NextJS: I'll be honest – I'm not proficient in NextJS yet. LaunchStack is my way of diving deep into this powerful framework. I'm learning as I go, and I'm excited to share this journey with you!
Building a Useful Tool: As I learn, I want to create something that could eventually help me (and maybe you!) build web projects more efficiently.
Challenging Myself: This is a pretty ambitious project for a NextJS beginner like me. But I believe in learning by doing, even if it means tackling something that seems a bit out of reach right now.
Sharing the Journey: By documenting this process, I hope to inspire other developers who are also at the beginning of their journey with new technologies.

Since I left my job, I have more time to dedicate to this project. My goal is to become proficient with NextJS by the end of it. I estimate it may take me around 6 months to a year to feel comfortable and confident working with NextJS, but I'm excited about the journey and the learning process.

What do I hope LaunchStack will include?

As I learn and grow, I'm aiming to incorporate these features and identify the technology to incorporate for each. For now, I just listed what I know based on my research.

| Feature | Technologies
(subject to change based on research) | | --- | --- | | Auth | AuthJS | | API Security / Rate Limiting | Unkey, Zuplo | | Theming | Tailwind, DaisyUI, Shadcn | | Payments / Subscriptions | Stripe, LemonSqueezy, RevenueCat | | Forms | Netlify, HubSpot Forms, Formspree | | Email Sending | Mailgun | | File Storage | Supabase, S3 | | ORM | Drizzle | | Database | PocketBase, SQLite
Postgres (Vercel, Supabase)
Redis (Vercel KV) | | Analytics | Google Analytics, Posthog, Plausible | | Deployment | Vercel, Netlify | | AI | Vercel AI SDK |

A Call for Support and Patience

This project is ambitious, especially given my current skill level with NextJS. There will be struggles, mistakes, and probably moments of frustration. But I'm committed to learning and growing through this process.

If you're an experienced NextJS developer, I'd love your advice and insights. If you're a beginner like me, maybe we can learn together! And if you're just curious about the process, I invite you to follow along on this journey.

For now, LaunchStack will not be open-sourced until it is decent enough to show the world. This is something I have wanted to do for a long time, inspired by projects like ShipFast, LaunchFast and DivJoy. As a beginner, it's hard to start building all these features from scratch, but I'm determined to make it happen.

I can't promise that LaunchStack will be ready for use anytime soon. What I can promise is an honest, open look at what it's like to tackle a big project as a way of learning a new technology.

Stay tuned for updates , lessons learned, and hopefully, steady progress towards a useful tool for all of us aspiring startup founders! You can subscribe here or follow me in Twitter, see you! I go online in Youtube or Twitch when I feel like doing live coding.

Let's learn, code, and grow together!

Integrating Generative AI with Real-Time Data from APIs - Groq, Python and Go

DonvitoCodes — Tue, 25 Jun 2024 08:08:33 GMT

In the world of conversational AI, the ability to integrate with external systems and retrieve information in real-time is a game-changer. Imagine being able to tap into the power of custom API calls, REST APIs, and other external systems to provide accurate and up-to-date responses to user questions.

This is precisely what Groq's function calling feature offers, allowing developers to harness the power of NLP processing to understand which function to call and retrieve the necessary information.

In this blog post, we'll explore how to use Groq's function calling feature to build conversational AI models that can interact with external systems, and demonstrate its capabilities using a simple example. We'll use an HTTP API written in Go to simulate an external API call which, ideally, will retrieve real data from a database. We'll break down the code into smaller parts, explaining each section in detail.

Architecture of what we'll build

Mock API in Go

Here is the code, just copy paste it into a main.go file

package main

import (
    "encoding/json"
    "net/http"
    "strings"
)

type CondoInfo struct {
    Price    string `json:"price"`
    Location string `json:"location"`
}

var condos = map[string]CondoInfo{
    "arton": {
        Price:    "Php 7,000,000",
        Location: "Katipunan, Quezon City",
    },
    "gold residences": {
        Price:    "Php 8,000,000",
        Location: "Near NAIA Terminal 1, Paranaque City",
    },
    "aruga mactan": {
        Price:    "Php 9,000,000",
        Location: "Mactan, Cebu",
    },
}

func getCondoLocation(w http.ResponseWriter, r *http.Request) {
    condoName := strings.ToLower(r.URL.Query().Get("condo_name"))
    condo, exists := condos[condoName]
    if !exists {
        http.Error(w, "Condo not found", http.StatusNotFound)
        return
    }

    response := map[string]string{"location": condo.Location}
    err := json.NewEncoder(w).Encode(response)
    if err != nil {
        return
    }
}

func getCondoPrice(w http.ResponseWriter, r *http.Request) {
    condoName := strings.ToLower(r.URL.Query().Get("condo_name"))
    condo, exists := condos[condoName]
    if !exists {
        http.Error(w, "Condo not found", http.StatusNotFound)
        return
    }

    response := map[string]string{"price": condo.Price}
    err := json.NewEncoder(w).Encode(response)
    if err != nil {
        return
    }
}

func main() {
    http.HandleFunc("/get_condo_location", getCondoLocation)
    http.HandleFunc("/get_condo_price", getCondoPrice)

    err := http.ListenAndServe(":8080", nil)
    if err != nil {
        return
    }
}

Run the API using the go command. This will run the API in http://localhost:8000

go run main.go

The API implements the following

get_condo_location endpoint:

curl "http://localhost:8080/get_condo_location?condo_name=Arton"

{"location":"Katipunan, Quezon City"}

get_condo_price endpoint:

curl "http://localhost:8080/get_condo_price?condo_name=Arton"

{"price":"Php 7,000,000"}

Let's start implementing!

This code is written in Python. This will respond to a query or prompt from a user and answer based on the data retrieved from an API. For this example, we will be using the Go API we created above to get the data to answer the ff. questions:

💡

What is the price of Arton?

💡

Where is the Arton condo?

from groq import Groq
import os
import json

client = Groq(api_key=os.getenv('GROQ_API_KEY'))
MODEL = 'llama3-70b-8192'

Here, we import the necessary libraries, including Groq, os, and json. We then set up a Groq client using an API key stored in an environment variable. The MODEL variable specifies the language model we'll be using.

Install the groq dependency using pip install.

pip install groq

Define Functions for the Tool

Ideally, responses from these functions should be from responses of your APIs. Right now, we will use our API we created earlier for demonstration purposes. Imagine that this function retrieves the price from a REST API. This is powerful because you can take advantage of NLP processing to understand which function to call. This is also available in OpenAI and this functionality works similar to it. But for this example, we will harness the power of Groq's fast inference!

def get_condo_price(condo_name):
    """Get the price of a condominium"""
    return condo_price_api_call(condo_name)

def get_condo_location(condo_name):
    """Get the location of a condominium"""
    return condo_location_api_call(condo_name)

Function 1.get_condo_price that takes a condominium name as input and returns the price of the condominium.

Function 2.get_condo_location that takes a condominium name as input and returns the location of the condominium.

API Calls to be used by the Functions

Then, we'll define the functions to call the external API which in this case is the API we created initially with Go. Feel free to change this to call your own APIs.

def condo_price_api_call(condo_name):
    url = f'http://localhost:8080/get_condo_price?condo_name={condo_name}'
    response = requests.get(url)
    if response.status_code == 200:
        data = response.json()
        return data.get('price')
    else:
        return None  # Handle error cases here if needed

def condo_location_api_call(condo_name):
    url = f'http://localhost:8080/get_condo_location?condo_name={condo_name}'
    response = requests.get(url)
    if response.status_code == 200:
        data = response.json()
        return data.get('location')
    else:
        return None

Running the Conversation

def run_conversation(user_prompt):
    ...

The run_conversation function takes a user prompt as input and sends it to the Groq model along with the available functions (get_condo_price and get_condo_location).

Step 1: Send the Conversation and Available Functions to the Model

    messages=[
        {
            "role": "system",
            "content": "You are a function calling LLM that uses the data extracted from the function and responds to "
                       "the user with the result of the function."
        },
        {
            "role": "user",
            "content": user_prompt,
        }
    ]
    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_condo_price",
                "description": "Get the price of a condo or condominium name",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "condo_name": {
                            "type": "string",
                            "description": "The name of the condominium or condo",
                        }
                    },
                    "required": ["condo_name"],
                },
            },
        },
         {
            "type": "function",
            "function": {
                "name": "get_condo_location",
                "description": "Get the location of a condo or condominium name",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "condo_name": {
                            "type": "string",
                            "description": "The name of the condominium or condo",
                        }
                    },
                    "required": ["condo_name"],
                },
            },
        }
    ]

We define the conversation and available functions to send to the model.

Step 2: Get the Model's Response

    response = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        tools=tools,
        tool_choice="auto",
        max_tokens=4096
    )

We get the model's response to the conversation and available functions.

This is how the response object looks like.


ChatCompletionMessage(
    content=None,
    role='assistant',
    function_call=None,
    tool_calls=[
        ChatCompletionMessageToolCall(
            id='call_2nvf',
            function=Function(
                arguments='{"condo_name":"Arton"}',
                name='get_condo_price'
            ),
            type='function'
        )
    ]
)

Step 3: Check if the Model Wanted to Call a Function

    response_message = response.choices[0].message
    tool_calls = response_message.tool_calls

    if tool_calls: #checks if tool_calls is an object
        ...

We check if the model wanted to call a function.

Step 4: Call the Function

        available_functions = {
            "get_condo_price": get_condo_price,
            "get_condo_location": get_condo_location,
        }
        messages.append(response_message)  # extend conversation with assistant's reply
        for tool_call in tool_calls: #loop through tool_calls and get the function to call
            function_name = tool_call.function.name
            function_to_call = available_functions[function_name]
            function_args = json.loads(tool_call.function.arguments)
            function_response = function_to_call(
                condo_name=function_args.get("condo_name")
            )
            messages.append(
                {
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": function_name,
                    "content": function_response,
                }
            )  # extend conversation with function response

We call the function and append the function response to the conversation.

Step 5: Get a New Response from the Model

        second_response = client.chat.completions.create(
            model=MODEL,
            messages=messages
        )  # get a new response from the model where it can see the function response
        return second_response.choices[0].message.content

We get a new response from the model where it can see the function response.

Step 6: Testing the Conversation

prompt = "What is the price of Arton?"
print('Price:', run_conversation(prompt))

prompt = "Where is the Arton condo?"
print('Location:', run_conversation(prompt))

We test the conversation by providing two user prompts:
"What is the price of Arton?" and "Where is the Arton condo?"

Step 7: Running the Python app

python main.py

You should see this response in your terminal.

Full Source Code

Here's is the full source code of the example. Just save it in a main.py file.

from groq import Groq
import os
import json
import requests

client = Groq(api_key=os.getenv('GROQ_API_KEY'))
MODEL = 'llama3-70b-8192'


def get_condo_price(condo_name):
    """Get the price of a condominium"""
    return condo_price_api_call(condo_name)


def condo_price_api_call(condo_name):
    url = f'http://localhost:8080/get_condo_price?condo_name={condo_name}'
    response = requests.get(url)
    if response.status_code == 200:
        data = response.json()
        return data.get('price')
    else:
        return None  # Handle error cases here if needed


def get_condo_location(condo_name):
    """Get the location of a condominium"""
    return condo_location_api_call(condo_name)


def condo_location_api_call(condo_name):
    url = f'http://localhost:8080/get_condo_location?condo_name={condo_name}'
    response = requests.get(url)
    if response.status_code == 200:
        data = response.json()
        return data.get('location')
    else:
        return None


def run_conversation(user_prompt):
    # Step 1: send the conversation and available functions to the model
    messages = [
        {
            "role": "system",
            "content": "You are a function calling LLM that uses the data extracted from the function and responds to "
                       "the user with the result of the function. Do not mention about anything about the tool call."
                       "Just respond with the answer to the user prompt."
        },
        {
            "role": "user",
            "content": user_prompt,
        }
    ]
    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_condo_price",
                "description": "Get the price of a condo or condominium name",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "condo_name": {
                            "type": "string",
                            "description": "The name of the condominium or condo",
                        }
                    },
                    "required": ["condo_name"],
                },
            },
        },
        {
            "type": "function",
            "function": {
                "name": "get_condo_location",
                "description": "Get the location of a condo or condominium name",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "condo_name": {
                            "type": "string",
                            "description": "The name of the condominium or condo",
                        }
                    },
                    "required": ["condo_name"],
                },
            },
        }
    ]
    response = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        tools=tools,
        tool_choice="auto",
        max_tokens=4096
    )

    response_message = response.choices[0].message
    tool_calls = response_message.tool_calls

    # Step 2: check if the model wanted to call a function
    if tool_calls:
        # Step 3: call the function
        # Note: the JSON response may not always be valid; be sure to handle errors
        available_functions = {
            "get_condo_price": get_condo_price,
            "get_condo_location": get_condo_location,
        }  # only one function in this example, but you can have multiple
        messages.append(response_message)  # extend conversation with assistant's reply
        # Step 4: send the info for each function call and function response to the model
        for tool_call in tool_calls:
            function_name = tool_call.function.name
            function_to_call = available_functions[function_name]
            function_args = json.loads(tool_call.function.arguments)
            function_response = function_to_call(
                condo_name=function_args.get("condo_name")
            )
            messages.append(
                {
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": function_name,
                    "content": function_response,
                }
            )  # extend conversation with function response
        second_response = client.chat.completions.create(
            model=MODEL,
            messages=messages
        )  # get a new response from the model where it can see the function response
        return second_response.choices[0].message.content


prompt = "What is the price of Arton?"
print('Price:', run_conversation(prompt))

prompt = "Where is the Arton condo?"
print('Location:', run_conversation(prompt))

Summary

In this blog post, we created an HTTP API using Go to simulate retrieval of data which was used to answer user questions or queries.

We explored the function calling feature of the Groq APIs, which allows us to call external functions to retrieve information. We defined two functions, get_condo_price and get_condo_location, and used the run_conversation function to send the conversation and available functions to the model.

We then checked if the model wanted to call a function, called the function, and appended the function response to the conversation. Finally, we tested the conversation with two user prompts. The function calling feature of the Groq APIs provides a powerful way to build conversational AI models that can interact with external systems.

If you're interested in learning more about integrating Go with Generative AI, follow my blog for more tutorials and insights. This is just the start!

I do live coding in Twitch and Youtube. You can follow me if you'd like to ask me questions when I go live. I also post in LinkedIn, you can connect with me there as well.

Interested in implementing AI within your company, you can reach out to me.

https://buymeacoffee.com/donvitocodes/e/260379

Not yet sure how can I help? Book a FREE discovery call with me.

https://buymeacoffee.com/donvitocodes/e/260160

Using Go to Translate Text using the Groq API (Fast Inference)

DonvitoCodes — Mon, 24 Jun 2024 09:50:41 GMT

In this blog post, I'll walk you through the process of using Go to translate text using the Groq's Fast Inference API.

What is Groq?

Groq is a new company that's making AI work much faster than ever before. They've created special computer parts called Language Processing Units (LPUs) that are really good at understanding and creating language. Imagine being able to generate so many words in a second - that's how fast Groq's technology is! It's way quicker than the computers we usually use for AI.

Groq vs ChatGPT

https://youtu.be/8UzW_AGX68g?si=9bZsn7lxxH88XwWy&t=16

This super fast speed is great for lots of things. It can help make:

Translation apps that work right away
Chatbots that talk to you without waiting
Trading programs that make decisions super fast

The best part? Groq made their fast APIs available for anyone to use.

Groq supports different open-source LLM models so it is a great alternative to OpenAI's GPT models.

By the end of this tutorial, you'll have a clear understanding of how to set up your environment, make Groq API requests, and handle the responses to translate text. Let's dive in!

Prerequisites

Before we start, ensure you have the following:

A basic understanding of the Go programming language.
Go installed on your machine. I am using the latest version Go 1.22.
An API key from the Groq Fast Inference API.

Getting Your Groq API Key

To interact with the Groq Fast Inference API, you'll need an API key. Follow these steps to get your API key:

Create an account in https://groq.com.
Sign up or log in to your Groq account.
Visit https://console.groq.com/keys. This page shows all your API keys.
Generate a new API key which we need for to access Groq's API. You should see this view after clicking Create API Key. Use a display name that you like.
Copy and the newly generated API key in a secure place, as you'll need it for the following steps.

Setting Up Your Go Environment

Create a new Go project and initialize a Go module:

mkdir go-groq
cd go-groq
go mod init go-groq

Next, create a file named main.go and add the following code:

package main

import (
    "errors"
    "fmt"
    "os"
)

func main() {

    apiKey := os.Getenv("GROQ_API_KEY")
    if apiKey == "" {
        err := errors.New("GROQ_API_KEY needs to be set as an environment variable")
        panic(err)
    }

    groqClient := &GroqClient{ApiKey: apiKey}
    textToTranslate := "Kim Ji-won was born on October 19, 1992, in Geumcheon District, Seoul, South Korea, and has an elder sister who is two years older than her. While still a teenager in 2007, she was scouted on the street and signed with an entertainment agency, she subsequently became a trainee for over three years while preparing for her debut. During her first year of junior high school, she spent six months to a year studying in Chicago, Illinois, United States, where her maternal relatives lived."

    systemPrompt := "you are a professional language translator. " +
        "only respond with the translated text. do not explain."
    prompt := fmt.Sprintf("translate this text to Tagalog: %s", textToTranslate)

    translatedText, err := groqClient.ChatCompletion(LLMModelLlama370b, systemPrompt, prompt)
    if err != nil {
        fmt.Println(err)
    }

    if translatedText != nil {
        fmt.Println(*translatedText)
    }
}

Implementing the GroqClient

Next, we'll implement the GroqClient that will handle communication with the Groq Fast Inference API. Create a new file named groq_client.go and add the following code:

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
)

const (
    apiBaseUrl = "https://api.groq.com/openai"
    SYSTEM     = "system"
    USER       = "user"

    LLMModelLlama38b       = "llama3-8b-8192"
    LLMModelLlama370b      = "llama3-70b-8192"
    LLMModelMixtral8x7b32k = "mixtral-8x7b-32768"
    LLMModelGemma7b        = "gemma-7b-it"
)

type GroqClient struct {
    ApiKey string
}

type GroqMessage struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type ChatCompletionRequest struct {
    Messages    []GroqMessage `json:"messages"`
    Model       string        `json:"model"`
    Temperature int           `json:"temperature"`
    MaxTokens   int           `json:"max_tokens"`
    TopP        int           `json:"top_p"`
    Stream      bool          `json:"stream"`
    Stop        interface{}   `json:"stop"`
}

type ChatCompletionResponse struct {
    Id      string `json:"id"`
    Object  string `json:"object"`
    Created int    `json:"created"`
    Model   string `json:"model"`
    Choices []struct {
        Index   int `json:"index"`
        Message struct {
            Role    string `json:"role"`
            Content string `json:"content"`
        } `json:"message"`
        Logprobs     interface{} `json:"logprobs"`
        FinishReason string      `json:"finish_reason"`
    } `json:"choices"`
    Usage struct {
        PromptTokens     int     `json:"prompt_tokens"`
        PromptTime       float64 `json:"prompt_time"`
        CompletionTokens int     `json:"completion_tokens"`
        CompletionTime   float64 `json:"completion_time"`
        TotalTokens      int     `json:"total_tokens"`
        TotalTime        float64 `json:"total_time"`
    } `json:"usage"`
    SystemFingerprint string `json:"system_fingerprint"`
    XGroq             struct {
        Id string `json:"id"`
    } `json:"x_groq"`
}

func (g *GroqClient) ChatCompletion(llmModel string, systemPrompt string, prompt string) (*string, error) {

    llm := llmModel

    if llmModel == "" {
        //default to llama8B
        llm = LLMModelLlama38b
    }
    groqMessages := make([]GroqMessage, 0)

    if systemPrompt != "" {
        systemMessage := GroqMessage{
            Role:    SYSTEM,
            Content: systemPrompt,
        }
        groqMessages = append(groqMessages, systemMessage)
    }

    if prompt != "" {
        userMessage := GroqMessage{
            Role:    USER,
            Content: prompt,
        }
        groqMessages = append(groqMessages, userMessage)
    } else {
        return nil, fmt.Errorf("prompt is required")
    }

    chatCompletionRequest := &ChatCompletionRequest{
        Messages:    groqMessages,
        Model:       llm,
        Temperature: 0,
        MaxTokens:   1024,
        TopP:        1,
        Stream:      false,
        Stop:        nil,
    }

    chatCompletionRequestJson, err := json.Marshal(chatCompletionRequest)
    if err != nil {
        return nil, err
    }

    //send http post request
    chatCompletionUrl := "/v1/chat/completions"
    finalUrl := fmt.Sprintf("%s%s", apiBaseUrl, chatCompletionUrl)

    req, err := http.NewRequest(http.MethodPost, finalUrl, bytes.NewBuffer(chatCompletionRequestJson))
    if err != nil {
        return nil, err
    }

    //set headers
    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", g.ApiKey))

    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        return nil, err
    }

    if resp.StatusCode != 200 {
        return nil, fmt.Errorf("unexpected status code: %d, reason: %s", resp.StatusCode, resp.Status)
    }

    defer func(Body io.ReadCloser) {
        err = Body.Close()
        if err != nil {
            fmt.Println("Error:", err)
        }
    }(resp.Body)

    body, err := io.ReadAll(resp.Body)
    if err != nil {
        return nil, err
    }

    chatCompletionResp := &ChatCompletionResponse{}

    err = json.Unmarshal(body, &chatCompletionResp)
    if err != nil {
        return nil, err
    }

    var content string
    if chatCompletionResp.Choices != nil && len(chatCompletionResp.Choices) > 0 {
        content = chatCompletionResp.Choices[0].Message.Content
    } else {
        return nil, fmt.Errorf("no choices")
    }

    return &content, nil
}

Breaking Down the GroqClient Code

Let's go through the GroqClient code step by step to understand how it works.

Constants and Structs

First, we define some constants and structs used throughout the code:

const (
    apiBaseUrl = "https://api.groq.com/openai"
    SYSTEM     = "system"
    USER       = "user"

    LLMModelLlama38b       = "llama3-8b-8192"
    LLMModelLlama370b      = "llama3-70b-8192"
    LLMModelMixtral8x7b32k = "mixtral-8x7b-32768"
    LLMModelGemma7b        = "gemma-7b-it"
)

type GroqClient struct {
    ApiKey string
}

type GroqMessage struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type ChatCompletionRequest struct {
    Messages    []GroqMessage `json:"messages"`
    Model       string        `json:"model"`
    Temperature int           `json:"temperature"`
    MaxTokens   int           `json:"max_tokens"`
    TopP        int           `json:"top_p"`
    Stream      bool          `json:"stream"`
    Stop        interface{}   `json:"stop"`
}

type ChatCompletionResponse struct {
    Id      string `json:"id"`
    Object  string `json:"object"`
    Created int    `json:"created"`
    Model   string `json:"model"`
    Choices []struct {
        Index   int `json:"index"`
        Message struct {
            Role    string `json:"role"`
            Content string `json:"content"`
        } `json:"message"`
        Logprobs     interface{} `json:"logprobs"`
        FinishReason string      `json:"finish_reason"`
    } `json:"choices"`
    Usage struct {
        PromptTokens     int     `json:"prompt_tokens"`
        PromptTime       float64 `json:"prompt_time"`
        CompletionTokens int     `json:"completion_tokens"`
        CompletionTime   float64 `json:"completion_time"`
        TotalTokens      int     `json:"total_tokens"`
        TotalTime        float64 `json:"total_time"`
    } `json:"usage"`
    SystemFingerprint string `json:"system_fingerprint"`
    XGroq             struct {
        Id string `json:"id"`
    } `json:"x_groq"`
}

Here, we define constants for API base URLs, message roles, and model names. The GroqClient struct holds the API key, while GroqMessage, ChatCompletionRequest, and ChatCompletionResponse structs define the request and response formats.

ChatCompletion Function

Let's break down the ChatCompletion function. This function calls the LLM which will do the translation of the text we send it to. You can use the different models supported by Groq. I have already listed the text-based models in the constants we previously defined.

💡

Check out the Groq Documentation for the list of models supported

func (g *GroqClient) ChatCompletion(llmModel string, systemPrompt string, prompt string) (*string, error) {
    // Determine the model to use
    llm := llmModel
    if llmModel == "" {
        llm = LLMModelLlama38b
    }

    // Create messages slice
    groqMessages := make([]GroqMessage, 0)

    // Add system message if provided
    if systemPrompt != "" {
        systemMessage := GroqMessage{
            Role:    SYSTEM,
            Content: systemPrompt,
        }
        groqMessages = append(groqMessages, systemMessage)
    }

    // Add user prompt message
    if prompt != "" {
        userMessage := GroqMessage{
            Role:    USER,
            Content: prompt,
        }
        groqMessages = append(groqMessages, userMessage)
    } else {
        return nil

, fmt.Errorf("prompt is required")
    }

    // Create request payload
    chatCompletionRequest := &ChatCompletionRequest{
        Messages:    groqMessages,
        Model:       llm,
        Temperature: 0,
        MaxTokens:   1024,
        TopP:        1,
        Stream:      false,
        Stop:        nil,
    }

    // Serialize request to JSON
    chatCompletionRequestJson, err := json.Marshal(chatCompletionRequest)
    if err != nil {
        return nil, err
    }

    // Construct the final URL
    chatCompletionUrl := "/v1/chat/completions"
    finalUrl := fmt.Sprintf("%s%s", apiBaseUrl, chatCompletionUrl)

    // Create HTTP request
    req, err := http.NewRequest(http.MethodPost, finalUrl, bytes.NewBuffer(chatCompletionRequestJson))
    if err != nil {
        return nil, err
    }

    // Set headers
    req.Header.Set("Content-Type", "application/json")
    req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", g.ApiKey))

    // Execute HTTP request
    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        return nil, err
    }

    // Check response status code
    if resp.StatusCode != 200 {
        return nil, fmt.Errorf("unexpected status code: %d, reason: %s", resp.StatusCode, resp.Status)
    }

    // Read response body
    defer func(Body io.ReadCloser) {
        err = Body.Close()
        if err != nil {
            fmt.Println("Error:", err)
        }
    }(resp.Body)

    body, err := io.ReadAll(resp.Body)
    if err != nil {
        return nil, err
    }

    // Parse response JSON
    chatCompletionResp := &ChatCompletionResponse{}
    err = json.Unmarshal(body, &chatCompletionResp)
    if err != nil {
        return nil, err
    }

    // Extract the translated text
    var content string
    if chatCompletionResp.Choices != nil && len(chatCompletionResp.Choices) > 0 {
        content = chatCompletionResp.Choices[0].Message.Content
    } else {
        return nil, fmt.Errorf("no choices")
    }

    return &content, nil
}

Determine the Model to Use: We start by setting the model to use for the request. If none is provided, we default to LLMModelLlama38b.

// Determine the model to use
llm := llmModel
if llmModel == "" {
    llm = LLMModelLlama38b
}

Create Messages Slice: We create a slice to hold the messages for the chat completion request.

// Create messages slice
groqMessages := make([]GroqMessage, 0)

Add System Message: If a system prompt is provided, we add it as a system message.

// Add system message if provided
if systemPrompt != "" {
    systemMessage := GroqMessage{
        Role:    SYSTEM,
        Content: systemPrompt,
    }
    groqMessages = append(groqMessages, systemMessage)
}

Add User Prompt Message: We add the user prompt as a message. If the prompt is empty, we return an error.

// Add user prompt message
if prompt != "" {
    userMessage := GroqMessage{
        Role:    USER,
        Content: prompt,
    }
    groqMessages = append(groqMessages, userMessage)
} else {
    return nil, fmt.Errorf("prompt is required")
}

Create Request Payload: We construct the request payload with the messages, model, and other parameters.

// Create request payload
chatCompletionRequest := &ChatCompletionRequest{
    Messages:    groqMessages,
    Model:       llm,
    Temperature: 0,
    MaxTokens:   1024,
    TopP:        1,
    Stream:      false,
    Stop:        nil,
}

Serialize Request to JSON: We serialize the request payload to JSON format.

// Serialize request to JSON
chatCompletionRequestJson, err := json.Marshal(chatCompletionRequest)
if err != nil {
    return nil, err
}

Construct the Final URL: We build the final URL for the API endpoint.

// Construct the final URL
chatCompletionUrl := "/v1/chat/completions"
finalUrl := fmt.Sprintf("%s%s", apiBaseUrl, chatCompletionUrl)

Create HTTP Request: We create a new HTTP POST request with the serialized JSON payload.

// Create HTTP request
req, err := http.NewRequest(http.MethodPost, finalUrl, bytes.NewBuffer(chatCompletionRequestJson))
if err != nil {
    return nil, err
}

Set Headers: We set the necessary headers, including the authorization header with the API key.

// Set headers
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", g.ApiKey))

Execute HTTP Request: We send the HTTP request using an HTTP client.

// Execute HTTP request
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
    return nil, err
}

Check Response Status Code: We check the response status code to ensure it's 200 (OK).

// Check response status code
if resp.StatusCode != 200 {
    return nil, fmt.Errorf("unexpected status code: %d, reason: %s", resp.StatusCode, resp.Status)
}

Read Response Body: We read the response body and defer the closing of the body.

// Read response body
defer func(Body io.ReadCloser) {
    err = Body.Close()
    if err != nil {
        fmt.Println("Error:", err)
    }
}(resp.Body)

body, err := io.ReadAll(resp.Body)
if err != nil {
    return nil, err
}

Parse Response JSON: We parse the response JSON into a ChatCompletionResponse struct.

// Parse response JSON
chatCompletionResp := &ChatCompletionResponse{}
err = json.Unmarshal(body, &chatCompletionResp)
if err != nil {
    return nil, err
}

Extract Translated Text: We extract the translated text from the response and return it.

// Extract the translated text
var content string
if chatCompletionResp.Choices != nil && len(chatCompletionResp.Choices) > 0 {
    content = chatCompletionResp.Choices[0].Message.Content
} else {
    return nil, fmt.Errorf("no choices")
}

return &content, nil

Running the Application

Before running the application, make sure to set the GROQ_API_KEY environment variable with your Groq API key. You can do this in your terminal:

export GROQ_API_KEY=your_api_key_here

Now, run your Go application:

go run main.go groq_client.go

If everything is set up correctly, you should see the translated text printed in your terminal.

Download Source Code in Github

The full source code for this tutorial is available in Github. Don't forget to ⭐ it.

https://github.com/donvito/learngo/tree/master/ai/go-groq

In this blog post, we've walked through the process of using Go to translate text using the Groq Fast Inference API. We covered how to set up your environment, make API requests, and handle the responses to perform text translation. By following these steps, you can easily integrate the Groq Fast Inference API into your Go applications for various language processing tasks. Happy coding!

You might want to check my other blog post about Groq Function calling

https://blog.donvitocodes.com/integrating-generative-ai-with-real-time-data-from-apis-groq-python-and-go

If you're interested in learning more about integrating Go with Generative AI, follow this blog for more tutorials and insights. This is just the start!

I do live coding in Twitch and Youtube. You can follow me if you'd like to ask me questions when I go live. I also post in LinkedIn, you can connect with me there as well.

Discovering Tech and AI

Using the Claude Agent SDK for Non-Coding Workflows

The Setup

Concept 1: Using MCPs for Data Retrieval

Concept 2: Sub-Agents for Specialized Tasks

Why This Matters

A Great Alternative to AI Agent Frameworks

Building a Java API connecting to LLMs with Spring AI and Ollama local models

Introduction

What We'll Build

Prerequisites

Step 1: Project Setup

1.1 Create the Project Structure

1.2 Create the pom.xml

1.3 Create the Main Application Class

Step 2: Define the Provider Architecture

2.1 Create the Provider Interface

2.2 Create the Provider Type Enum

2.3 Create the Provider Registry

Step 3: Implement the Ollama Provider

3.1 Create the Ollama Service

3.2 Create Configuration

Step 4: Create the REST API

4.1 Create the Controller

Step 5: Configure the Application

Step 6: Test the Application

6.1 Start Ollama

6.2 Run the Application

6.3 Test the Endpoints

List Available Providers

Send a Chat Request

Step 7: Adding New Providers

7.1 Add the Dependency

7.2 Create the Provider Implementation

7.3 Add to Provider Type Enum

7.4 Configure in application.properties

Advanced Features

Error Handling

Model Selection

Provider Discovery

Best Practices

Conclusion

Java-Based AI Solutions for Enterprises: Viability, Use Cases, and Market Outlook

Introduction: Java’s Role in Enterprise AI

Enterprise Needs and AI Adoption Trends

AI Use Cases for Enterprises (Text and Vision)

NLP and Text-Based Use Cases

Computer Vision Use Cases

Packaging and Integrating a Java AI Solution for Enterprise

Success Stories and Case Studies

Future Outlook and Opportunities

9 Ways How AI can be Useful for Emails

1. Drafting Replies Quickly

2. Adapting Tone and Style

3. Handling Different Types of Emails

4. Saving Time With Long Emails

5. Maintaining Consistency

6. Prioritizing Responses

7. Overcoming Writer’s Block

8. Multilingual Replies

9. Customizing Length and Detail

Introducing OpenAI o3-mini: A New Frontier in Cost-Effective STEM Reasoning

My Top AI Coding Tools

AI Coding Copilots

My AI Development Stack

Cursor AI for Interative Development

Bolt.new - a great starting point!

Vercel v0 - Generate UI Components

Codeium AI Code Completion

Using Claude and ChatGPT for Coding

Claude's Artifacts Feature

The Future of Development

Vision Agent: Transforming Business Through AI-Powered Computer Vision

What is Vision Agent?

Main Features

Simplified Development Process

Smart Tools and Automation

Vision AI Technology's Potential Use Cases for Business

Manufacturing & Quality Control

Healthcare & Medical Services

1.2 Create the `pom.xml`