Skip to main content
pcollins.tech
← All work

Personal project2024

Rare Tools - Japanese Tool Marketplace

Full Stack Development

Rare Tools is a specialised e-commerce platform that connects enthusiasts with premium Japanese tools. The platform features an automated scraping system that collects data from various Japanese marketplaces, processes it, and presents it through a modern, user-friendly interface. The application emphasizes performance, SEO, and user experience while maintaining data accuracy and real-time availability.

  • Next.js
  • TypeScript
  • Tailwind CSS
  • React Server Components
  • Node.js
  • AWS SDK
  • Puppeteer
  • OpenAI API
  • Vercel Analytics
Rare Tools - Japanese Tool Marketplace screenshot

5+

Product Categories

Different categories of Japanese tools available

3+

Data Sources

Japanese marketplaces integrated

The short version

What it does

A modern e-commerce platform for rare Japanese tools, featuring a Next.js frontend and an AI-powered data pipeline that scrapes, cleans and enriches listings from multiple marketplaces using the OpenAI API.

  • Modern e-commerce platform built with Next.js and App Router
  • Automated scraping system for Japanese tool marketplaces
  • SEO-optimised with Next.js Metadata
  • Server-side rendering with React Server Components
  • Responsive design with Tailwind CSS
  • Dark/Light mode support

The hard parts

What made it difficult, and what I did about it

The problems

  • Handling multiple Japanese marketplace data sources
  • Maintaining data consistency and accuracy
  • Implementing real-time price and availability updates
  • Managing complex product variations and specifications
  • Ensuring reliable image processing and storage

The answers

  • Built a scraping system using Puppeteer and TypeScript
  • Implemented data validation and normalization pipelines
  • Used AWS S3 for reliable image storage
  • Created a modular architecture for easy marketplace integration
  • Employed OpenAI API for enhanced data processing

What came of it

  • Successfully aggregated products from multiple Japanese marketplaces
  • Achieved fast page load times with Next.js optimisation
  • Implemented responsive design for all device types
  • Created an intuitive search and filtering system

Under the bonnet

How it is put together

The application follows a modern microservices architecture with a clear separation between the frontend e-commerce platform and the backend scraping system. It uses Next.js for the frontend and Node.js for the backend scraping services.

  • Next.js Frontend Application
  • TypeScript-based Scraping Service
  • AWS S3 Image Storage
  • OpenAI Integration for Data Processing
  • Vercel Analytics for Monitoring

Frontend

Framework
Next.js with App Router
State
React Server Components
Styling
Tailwind CSS with shadcn/ui components
  • Server-side rendering
  • Dynamic OG images
  • Automatic dark/light mode
  • SEO optimisation
  • Responsive design

Backend

Framework
Node.js with TypeScript
Database
JSON-based data storage
API
REST API with Express
  • Automated scraping system
  • Data normalization pipeline
  • Image processing and storage
  • OpenAI integration
  • Error handling and retry mechanisms

Infrastructure

Hosting
Vercel
Deployment
Vercel CI/CD
Monitoring
Vercel Analytics
  • Edge Runtime support
  • Automatic deployments
  • Performance monitoring
  • Error tracking

My part in it

Full Stack Developer

Team size: 1 developer

  • Frontend development
  • Backend development
  • Data scraping implementation
  • Infrastructure management
  • Performance optimisation

Got something like this in mind?

Tell me what you're trying to build. It's a free call, and if I'm not the right person for it I'll say so.

Rather write it down? Send a message instead.