Personal project2024
Rare Tools - Japanese Tool Marketplace
Full Stack Development
Rare Tools is a specialised e-commerce platform that connects enthusiasts with premium Japanese tools. The platform features an automated scraping system that collects data from various Japanese marketplaces, processes it, and presents it through a modern, user-friendly interface. The application emphasizes performance, SEO, and user experience while maintaining data accuracy and real-time availability.
- Next.js
- TypeScript
- Tailwind CSS
- React Server Components
- Node.js
- AWS SDK
- Puppeteer
- OpenAI API
- Vercel Analytics

5+
Product Categories
Different categories of Japanese tools available
3+
Data Sources
Japanese marketplaces integrated
The short version
What it does
A modern e-commerce platform for rare Japanese tools, featuring a Next.js frontend and an AI-powered data pipeline that scrapes, cleans and enriches listings from multiple marketplaces using the OpenAI API.
- Modern e-commerce platform built with Next.js and App Router
- Automated scraping system for Japanese tool marketplaces
- SEO-optimised with Next.js Metadata
- Server-side rendering with React Server Components
- Responsive design with Tailwind CSS
- Dark/Light mode support
The hard parts
What made it difficult, and what I did about it
The problems
- Handling multiple Japanese marketplace data sources
- Maintaining data consistency and accuracy
- Implementing real-time price and availability updates
- Managing complex product variations and specifications
- Ensuring reliable image processing and storage
The answers
- Built a scraping system using Puppeteer and TypeScript
- Implemented data validation and normalization pipelines
- Used AWS S3 for reliable image storage
- Created a modular architecture for easy marketplace integration
- Employed OpenAI API for enhanced data processing
What came of it
- Successfully aggregated products from multiple Japanese marketplaces
- Achieved fast page load times with Next.js optimisation
- Implemented responsive design for all device types
- Created an intuitive search and filtering system
Under the bonnet
How it is put together
The application follows a modern microservices architecture with a clear separation between the frontend e-commerce platform and the backend scraping system. It uses Next.js for the frontend and Node.js for the backend scraping services.
- Next.js Frontend Application
- TypeScript-based Scraping Service
- AWS S3 Image Storage
- OpenAI Integration for Data Processing
- Vercel Analytics for Monitoring
Frontend
- Framework
- Next.js with App Router
- State
- React Server Components
- Styling
- Tailwind CSS with shadcn/ui components
- Server-side rendering
- Dynamic OG images
- Automatic dark/light mode
- SEO optimisation
- Responsive design
Backend
- Framework
- Node.js with TypeScript
- Database
- JSON-based data storage
- API
- REST API with Express
- Automated scraping system
- Data normalization pipeline
- Image processing and storage
- OpenAI integration
- Error handling and retry mechanisms
Infrastructure
- Hosting
- Vercel
- Deployment
- Vercel CI/CD
- Monitoring
- Vercel Analytics
- Edge Runtime support
- Automatic deployments
- Performance monitoring
- Error tracking
My part in it
Full Stack Developer
Team size: 1 developer
- Frontend development
- Backend development
- Data scraping implementation
- Infrastructure management
- Performance optimisation
Got something like this in mind?
Tell me what you're trying to build. It's a free call, and if I'm not the right person for it I'll say so.
Rather write it down? Send a message instead.
