>

07/2024 ~ Present

Jujiang AI Multilingual Learning Platform

Full-Stack Development
Multilingual Learning
AI Applications
Speech Recognition
Active
Featured

An AI multilingual learning platform developed in collaboration with Jujiang, combining AI conversation, speech recognition, and learning feedback for English, Japanese, Korean, and other languages

Jujiang AI Multilingual Learning Platform

Tech Stack

React 19

Frontend framework with Vite for building modern SPA interfaces

FastAPI

Python async API framework handling backend business logic

PostgreSQL

Relational database storing user and learning records

SQLAlchemy 2.0

Fully async ORM with Alembic for database migration management

OpenAI API

GPT-powered multilingual conversation engine and content evaluation

Azure Speech Services

Multilingual speech-to-text (STT) and speaking analysis

Docker

Containerized deployment supporting multiple environment configurations

Tailwind CSS

Utility-first CSS framework

Project Overview

Developed in collaboration with Jujiang, this AI-driven multilingual learning platform supports scenario-based conversation and speaking practice in English, Japanese, Korean, and other languages. It combines OpenAI GPT for dialogue and content evaluation with Azure Speech Services for speech recognition and speaking analysis, then produces detailed learning reports.

The backend uses a fully async FastAPI architecture with a PostgreSQL database. The frontend is built with React 19 + Vite, deployed in containerized multi-environment configurations via Docker Compose.

Core Features

AI Real-Time Conversation Practice

A multilingual conversation engine powered by OpenAI GPT, supporting scenario-based and assessment modes with difficulty adjusted to the target language and learner level.

Multilingual Speech Recognition and Speaking Assessment

Integrates Azure Speech Services for real-time speech-to-text, analyzing speaking performance for each language and providing detailed feedback on a 50-60-80-100 scale.

Multilingual, Multi-Level Content

Organizes learning content by language, level, and scenario. Each level contains a structured conversation flow (ChatFlow) driven by JSON configuration.

Learning Analytics Reports

Records scores, speaking feedback, and improvement trends for each session, giving learners and administrators cross-language learning analytics.

Admin Dashboard

Administrators can manage learning records, create and edit learning levels, view assessment group data, and monitor system status through the admin panel.

Technical Architecture

Frontend

Built on React 19 with Vite, using React Router v7 for routing, Tailwind CSS for a custom-themed interface (purple primary + orange accent), and Axios for API communication.

Backend

Uses the fully async FastAPI framework with SQLAlchemy 2.0 and the asyncpg driver for async database operations, Alembic for migration management, and Poetry for Python dependency management.

AI and Speech Services

Conversation engine and content evaluation use OpenAI API (GPT + TTS). Speech recognition integrates Azure Cognitive Services Speech SDK. Audio processing uses SoundFile and PyDub for format conversion.

Authentication and Security

JWT token authentication with bcrypt password hashing, supporting a dual-role system for administrators and regular users.

Deployment

Docker Compose multi-environment deployment (Local / Dev / Prod) with Nginx reverse proxy, PostgreSQL database container, and PgWeb management interface.

Database Design

Uses PostgreSQL with SQLAlchemy 2.0 (async), managing learning record lifecycles through a state-machine pattern — from creation, conversation, and completion through to scoring — fully tracking every learning session.

Table Description Key Fields
users Learner accounts (UUID-identified) id, name, email
managers Administrator accounts (password stored as hash) id, username, password_hash

API Design

RESTful API designed with FastAPI, fully async to support real-time voice conversation scenarios:

Conversation and Voice /api/records

Create learning records, send voice messages, receive AI replies, speech-to-text processing

Learning Levels /api/chat-levels

Level CRUD, ChatFlow JSON configuration management, batch import (Excel to API)

Evaluation System /api/evaluate

AI speaking assessment, content accuracy analysis, tiered feedback generation

User Management /api/users / /api/managers

Learner and administrator account management, JWT authentication, token quota tracking

Utility Services /api/tools

Audio format conversion, TTS speech synthesis, system health check

Deployment Architecture

Multi-Environment Docker Compose Configuration

Supports three deployment environments — Local (direct connection), Development (Nginx + Hot Reload), and Production (Nginx reverse proxy + optimized build) — all including a PostgreSQL database container and PgWeb database management interface.

Service Components

Frontend (React SPA), Backend (FastAPI + Uvicorn), PostgreSQL 15, Nginx reverse proxy, PgWeb database management, shared audio file volume.