Disaster Response Assistant

A multimodal AI applet built for the Google AI Studio Challenge - demonstrating the power of Gemini's multimodal content understanding and generation capabilities.

This applet fully meets all requirements of the "Build and Deploy a Multimodal Applet" challenge:

✅ Built on Google AI Studio - Developed using Google AI Studio's development environment and APIs
✅ Deployed using Cloud Run - Production deployment on Google Cloud Run for scalability and reliability
✅ Multimodal Functionality - Implements multiple Gemini capabilities:

• Gemini 2.5 Flash for multimodal image and text understanding
• Imagen 4.0 for AI-generated "before disaster" visualizations
• Gemini Chat API for context-aware conversational support

The Disaster Response Assistant is a web application designed to provide immediate, AI-powered support to individuals in disaster-affected areas. In the chaotic aftermath of an earthquake, flood, or fire, getting clear, actionable information is critical for safety. This applet addresses that need by allowing users to quickly capture and send images and text descriptions of damage to their surroundings.

It solves the crucial problem of rapid situational assessment. Instead of waiting for emergency services who may be overwhelmed, users can get an instant analysis of their situation, including:

• A clear assessment of the structural damage.
• The severity level of the situation (from Low to Critical).
• A prioritized list of immediate, actionable safety steps.

The experience it creates is one of empowerment and reassurance during a highly stressful time. By transforming a user's phone into a powerful diagnostic tool, it helps reduce panic, provides a clear path forward, and enables users to take control and secure their immediate safety.

Live Applet: [Link to Your Deployed Applet Here]
Cloud Run Deployment: Hosted on Google Cloud Run with auto-scaling capabilities
Repository: [GitHub Repository Link]

1. Damage Reporting Interface: The clean, intuitive UI for uploading multiple images and adding a voice or text description.

Awaiting Screenshot

2. Comprehensive Analysis Results: The main results screen displaying the severity, damage assessment, and actionable guidance.

Awaiting Screenshot

3. Before & After Comparison: The powerful visual comparison showing the user's photo next to an AI-generated image of the location before the disaster.

Awaiting Screenshot

4. Interactive Follow-up Chat: The conversational AI chatbot that helps users with specific follow-up questions.

Awaiting Screenshot

• Platform: Google Cloud Run (containerized deployment)
• Scaling: Auto-scaling based on traffic with 0-to-N instances
• Runtime: Node.js with Express.js backend
• Frontend: React with TypeScript, served as static assets
• Build Process: Docker containerization with multi-stage builds

The entire application is orchestrated around the powerful multimodal capabilities of the Gemini API. I did not just use it for a single task, but created a chain of AI-driven operations to deliver a comprehensive user experience.

1. Multimodal Analysis (gemini-2.5-flash): The core of the app uses gemini-2.5-flash to process a complex, multimodal input: multiple user-uploaded images and a text description. I configured the model to use JSON Mode with a strict responseSchema. This is a critical best practice that ensures the AI's output is always structured, reliable, and can be directly used to populate the UI without risky parsing of natural language. A systemInstruction primes the model to act as a disaster response expert, ensuring the tone and content are appropriate.

2. Text-to-Image Generation (imagen-4.0-generate-001): To provide a powerful visual context of the damage, one of the fields in the structured JSON response from Gemini is a beforeImagePrompt. This prompt, created by the analysis model, is then fed directly into the imagen-4.0-generate-001 model to generate a realistic photo of the location before the disaster. This creates a seamless AI workflow from analysis to visualization.

3. Conversational AI (Gemini Chat API): For personalized support, I used the Gemini Chat API (ai.chats.create). The chat session is initialized with the context from the initial damage assessment. This makes the chatbot instantly aware of the user's situation. All responses from the chatbot are streamed to the UI, creating a dynamic, real-time conversational experience and showing the user information as soon as it's available.

The app is built on a foundation of multimodality, which dramatically enhances its utility and user experience in a crisis scenario.

• Image and Text Fusion for Superior Understanding: The app's primary input is multimodal. By analyzing images and text together, the AI gains a much deeper, more contextual understanding than it could from either modality alone. For example, the AI can correlate a user's text ("I hear cracking sounds") with a visual of a hairline fracture in a wall, leading to a more accurate severity assessment. This fusion is key to the app's effectiveness.

• Analysis-to-Visualization Workflow: The app doesn't just understand multimodal input; it generates multimodal output. The "Before Disaster" visualization is a prime example. The AI first sees and reads about a damaged scene, then it imagines and creates an image of that same scene in an undamaged state. This powerful feature gives users an immediate and visceral understanding of the extent of the damage.

• Visually-Grounded Conversation: The follow-up chatbot is more than a simple Q&A bot. Because its context is derived from the initial visual analysis, its answers are grounded in the user's actual environment. If a user asks, "Is that crack dangerous?", the AI's response is informed by the picture of the crack the user provided, making the guidance highly relevant and personal.

This applet goes beyond basic AI demos to address a critical real-world challenge: immediate disaster response assessment. In emergency situations, traditional response systems are often overwhelmed, leaving individuals without crucial safety information. The Disaster Response Assistant fills this gap by:

• Democratizing Expert Assessment: Transforms any smartphone into a structural damage assessment tool
• Reducing Response Time: Provides instant analysis instead of waiting hours for professional assessment
• Enabling Informed Decision-Making: Gives users concrete, prioritized actions based on their specific situation
• Supporting Emergency Services: Generates structured damage reports that can be shared with first responders

1. Cross-Modal Analysis: Combines visual damage assessment with textual context (sounds, smells, environmental factors) for comprehensive understanding
2. Temporal Visualization: Uses AI to reconstruct "before disaster" scenes, helping users understand damage extent
3. Context-Aware Conversation: Chatbot responses are grounded in the user's actual visual environment
4. Progressive Disclosure: Information is revealed in stages (assessment → visualization → conversation) to prevent cognitive overload during crisis

• Batch Image Upload: Users can upload multiple photos for a comprehensive review.
• Textual Context: A textarea allows users to add crucial context to the visual data.
• AI Damage Assessment: Structured JSON output provides a detailed assessment.
• Severity Level Classification: Damage is categorized as Low, Medium, High, or Critical.
• Actionable Safety Guidance: A clear, prioritized list of next steps for user safety.
• AI "Before Disaster" Visualization: A generated image shows the scene pre-disaster.
• Interactive Chatbot: A streaming, context-aware chat for follow-up questions.
• Downloadable Reports: Users can save the analysis and "before" image for offline use.

• Structured AI Output: Used responseSchema (JSON Mode) for robust, predictable, and error-free communication between the AI and the frontend.
• Clear State Management: Leveraged React's state management to handle loading, error, progress, and result states, providing immediate and clear UI feedback.
• Component-Based Architecture: The UI is built with modular, reusable React components, promoting clean code and maintainability.
• Asynchronous Flow Control: All API calls are handled with async/await and enclosed in try...catch blocks for graceful error handling.
• User-Centric Loading: Loading spinners and dynamic progress messages are displayed during API calls to manage user expectations.
• Streaming for UX: Chatbot responses are streamed to the UI to provide a responsive, real-time feel.
• Accessibility: Key interactive elements include aria-label attributes to ensure usability for users with screen readers.
• Responsive Design: The UI is fully responsive and accessible across devices, from mobile phones to desktops, using Tailwind CSS.
• Code Organization: Logic is separated into services (geminiService), utilities (fileUtils, downloadUtils), components, and types for a clean and scalable codebase.

• Frontend: React 18 + TypeScript + Tailwind CSS
• Backend: Node.js + Express.js
• AI Services: Google AI Studio APIs (Gemini 2.5 Flash, Imagen 4.0, Chat API)
• Deployment: Google Cloud Run with Docker containerization
• Build Tools: Vite for frontend bundling, Docker for containerization

// Multimodal analysis with structured output
const analysis = await gemini.generateContent({
  model: 'gemini-2.5-flash',
  contents: [{ parts: [imageData, textPrompt] }],
  generationConfig: {
    responseMimeType: 'application/json',
    responseSchema: damageAssessmentSchema
  }
});

// Streaming chat responses
const chatStream = await gemini.streamGenerateContent({
  model: 'gemini-2.5-flash',
  contents: conversationHistory
});

• Memory: 2GB for handling image processing
• CPU: 2 vCPU for concurrent request handling
• Concurrency: 100 requests per instance
• Timeout: 300 seconds for complex AI operations
• Environment Variables: Secure API key management

• Image Compression: Client-side image optimization before upload
• Lazy Loading: Progressive component loading for faster initial render
• Caching: Response caching for repeated analysis requests
• Error Boundaries: Graceful degradation for API failures

• Audio Analysis: Integration with Gemini's audio understanding for sound-based damage assessment
• Video Processing: Real-time video analysis for dynamic damage evaluation
• Offline Capabilities: Progressive Web App features for areas with limited connectivity
• Multi-language Support: Localization for global disaster response
• Integration APIs: Webhooks for emergency services and insurance companies