Machine Building Project Page¶
Slide¶
Video¶
Group Assignment Overview¶
This is the group assignment of the week.
- design a machine that includes mechanism+actuation+automation+application
- build the mechanical parts and operate it manually
- document the group project and your individual contribution
- actuate and automate your machine
- document the group project and your individual contribution
0. Who Did What¶
- Shintaro Ito: project management, AI model, camera module, and presentation video
- Yuya Tokuyama: software, server, and documentation
- Naoki Hayashi: hardware, assembly, and presentation slide
1. Machine to Make: RYOAN-JI XY 2025¶
In our group project, we decided to create the machine “RYOAN-JI XY 2025,” based on a CoreXY module, which uses an LLM to generate a desktop karesansui and haiku inspired by input images.
Below is our group project’s initial proposal.
If you check the Slide and Video at the top of the page, you’ll see that we were able to bring most of it to life!
Title of the work: “Memory Landscapes”
Concept
- When the audience inputs their past memories or emotions, the XYZ arm creates patterns resembling terrains or ripples on the surface of sand or powder. Over time, new memories from other audience members reconstruct the patterns, overwriting and erasing the previous ones—bringing to life the poetic message that “memories shift like landscapes.” This generative installation allows people to experience this ephemeral transition.
Expression Structure
Visual
- The XYZ arm draws patterns on white sand (or powder) spread across the base of a frame.
- Unique pattern generation intertwines randomness and structure, modulated by memory parameters.
- Lighting from above vividly highlights the shadows and contours, unveiling a beautiful landscape.
Sound (Optional)
- Slow ambient sounds (wind, water, voices) play during the drawing process.
Interactivity
- Audience members input “memories,” “keywords,” or “images” via smartphones or devices.
- The arm interprets these inputs to create expressions like “ripples,” “vortices,” “terraces,” and “valleys.”
- The completed landscapes are preserved for a certain period before being overwritten with new memories.
2. Mechanical and Machine Design¶
2-1. Machine design: CoreXY¶
CoreXY is a motion system that uses two stationary motors and a crossed belt arrangement to achieve precise, lightweight, and fast two-axis movement with minimal mechanical complexity.
2-2. Mechanical design based on UrumbotXY 2.0¶
We started with Quentin Bolsee’s UrumbotXY 2.0, adopting the CoreXY mechanism and adding features for Karesansui art. Below are several changes from Urumbot:
- Switched to a non-grooved aluminum frame, which required adjustments for stability.
- Added eccentric nuts and extra wheels for smoother movement.
- Built custom brackets to hold everything together securely.
2-3. Assembly¶
-
Assemble the carriage
- Because of the height of the eccentric nuts and V-wheel, we added some spacers made of acrylic board to adjust the height of the other wheels.
-
Assemble the X-frame
-
Install the X-frame into the Y-frame
Don’t forget to install T-nuts
Be sure to install the correct number of T-nuts before assembling the frames!
-
Assemble the remaining main frame and Z-frames while securing 90 degree angle
- Fix and wrap the 2 wires
- Assemble the Karesansui sand field
- Mount the camera modules and endstops
- Wiring and covering in the aluminium frames
Before building all functions, we quickly assembled the main frame and motors to manually test the CoreXY mechanism.
2-3. Calibration of movements¶
We tested how much the carriage actually moves when controlling a parameter in the PC program. By drawing a ‘50 x 50’ square using a pen attached to the carriage, we measured an actual size: 80 x 80 mm. This means the system moves at 8/5 = 1.6 of the programmed scale. Therefore, by multiplying the parameter by 5/8 = 0.625, the units within the program were adjusted to match mm.
2-5. Rake calibration based on sand behavior¶
The rake was initially too high and didn’t press into the sand enough, so I added nuts between the ball joint and the rake to lower it.
2-4. BOM¶
The RYOAN-JI XY 2025 is built using standard, easily accessible components, combined with locally fabricated parts made using 3D printing and laser cutting, all detailed in the BOM below.
Electronic components¶
| Item | Details | Qty | Remarks |
|---|---|---|---|
| Stepper Motor | NEMA17 | 3 | Motor for Y-axis |
| Limit Switch | Normally Open | 3 | XY axis and trigger |
| USB 3.0 Hub | Amazon | 1 | |
| USB Type C cable | Amazon | 4 | For 3 motors and a ESP32S3 |
| XIAO ESP32S3 Sense | Seeed | 1 | |
| XIAO RP2040 | Seeed | 3 | For 3 motors |
| Modular-Things CoreXY | 2024 Bootcamp | 3 |
Mechanical components¶
| Item | Details | Qty | Remarks |
|---|---|---|---|
| V-Slot Wheel | 16 | For all axis motion | |
| Bearing | D16xd5xH5 | 4 | Motor pulley |
| M5 eccentric nut | 6 | Carriage | |
| Non-stretch Fishing Line | Amazon: 9KM DWLIFE Kevlar Cord, Load Capacity 22kg~680kg | 1 |
Structural components¶
| Item | Details | Qty | Remarks |
|---|---|---|---|
| 20*20 Aluminum frame (HFS5-2020-390) | 390mm | 2 | Frame base Y |
| 20*20 Aluminum frame (HFS5-2020-350) | 350mm | 2 | Frame base X |
| 20*20 Aluminum frame (HFS5-2020-346) | 346mm | 1 | X-axis carriage |
| 20*20 Aluminum frame (HFS5-2020-175) | 175mm | 5 | Foot and camera |
Fasteners¶
| Type | Details | Quantity | Remarks |
|---|---|---|---|
| M5 Socket Head Screw | 8mm | 61 | General frame & assemblies |
| 30mm | 6 | Pulley, captain | |
| 40mm | 4 | X-axis carriage | |
| 50mm | 16 | XY-axis carriage | |
| M5 Nut | 16 | Carriage, pulley | |
| M5 washer | 6 | ||
| M5 spring washer | 16 | For pulleys and wheels | |
| M5 T-Slot | 70 | Frame and all axis assemblies | |
| M5 Acrylic Spacer | 5mm | 36 | Pulley and eccentric nuts |
| 3mm | 8 | Carriage | |
| M3 Socket Head Screw | 8mm | 16 | Stepper motor, ESP32S3 |
| 15mm | 2 | Rakes | |
| M3 nut | 8 | Fishing line holder | |
| M3 washer | 8 | Fishing line holder | |
| M2 Socket Head Screw | 12mm | 6 | Limit switch |
| M2 nut | 6 | Limit switch | |
| M3 Set screw | 8 | Pulleys |
Zen components¶
| Type | Details | Quantity | Remarks |
|---|---|---|---|
| Sand | Amazon: アマゾン川源流の白砂 3kg (約1.8L) | 1 |
2-3. Fabricated Components¶
We designed some parts from scratch: rake holders, a sand tray with LED lighting, camera mounts, and sensor brackets, was made from PLA filament, MDF and acrylic board, using 3D printing and laser cutting.
-
3D print / lasercut parts: Ryoan-Ji_XY_2025_v1_BOM.stp.zip
-
Assembled (*Includes imported models from Urumbot 2.0. Some purchased parts such as bolts are not included.)
3. LLM(Gemini) prompts¶
Ito-san, who conceptualized the machine “RYOAN-JI XY 2025,” which uses an LLM to create a desktop karesansui and haiku inspired by input images, also crafted the LLM (Gemini) prompts. Ito-san was in charge of overall project management, bringing fascinating concepts and wonderful prompts to life. Thank you, Ito-san!
3-1. Karesansui prompt¶
// Set API key (in actual use, retrieve it safely from environment variables, etc.)
const API_KEY = "YOUR_API_KEY";
const MODEL = "gemini-1.5-pro-latest"; // Model suitable for creative purposes
// Request configuration
async function generateKaresansuiSVG(imageData) {
// API endpoint
const endpoint = `https://generativelanguage.googleapis.com/v1beta/models/${MODEL}:generateContent?key=${API_KEY}`;
// Build request body
const requestBody = {
contents: [
{
parts: [
{
text: "Generate SVG code representing karesansui sand patterns based on this image."
},
{
inlineData: {
mimeType: "image/jpeg", // Change according to image type
data: imageData // Base64 encoded image data
}
}
]
}
],
// System instruction
systemInstruction: {
parts: [
{
text: `#ROLE
You are an expert at abstracting the impression of images and converting them into geometric patterns resembling mental landscapes, and you are proficient in generating SVG code.
# TASK
Abstract the provided image, convert it into geometric patterns resembling a mental landscape, and generate SVG code.
# TECHNICAL REQUIREMENTS
1. **Simplification:** Extract and simplify the key features and contours from the image while maintaining its essential character.
2. **Continuity:** Create continuous, unbroken lines that could be drawn by a mechanical rake or robotic arm in a single motion where possible.
3. **Only geometric shapes:** Only geometric shapes (e.g., circles, triangles, straight lines).
4. **Traditional Aesthetics:** Incorporate traditional karesansui design elements:
- Parallel ripple patterns (波紋, hamon) to represent water
- Straight lines to create clarity and structure
- Gentle curves to represent natural flow
- Asymmetrical balance (非対称の調和)
- Do not include text
5. **Technical Specifications:**
- Optimized for a vertical size of 226 mm × 240 mm
- Use a clear coordinate system with viewBox="0 0 226 240"
- Maintain proper scaling and proportions
- Ensure all lines have appropriate stroke width (typically 0.5-1.0px)
- Use only stroke paths (no fills)
# OUTPUT FORMAT
You must output ONLY the raw SVG code with no additional text, explanations, comments, markdown formatting, or code block markers(```svg ```).
The SVG code must:
- Start with the opening `< svg > ` tag including xmlns attribute and viewBox specification
- End with the closing `</svg> ` tag
- Be properly formatted and valid
- Be immediately renderable in a web browser or vector editing software without any modifications
# IMPORTANT
Do not include any explanatory text, introductions, or conclusions.
Do not wrap the SVG code in backticks or markdown code blocks.
Return only the SVG code itself, nothing else.
Example of expected output format:
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 210 297" width="210mm" height="297mm">
<path d="M10,10 C20,20 30,10 40,10" stroke="black" stroke-width="0.5" fill="none"/>
<!-- Additional paths as needed -->
</svg>`
}
]
},
// Generation settings
generationConfig: {
temperature: 0.85, // Higher temperature setting for creative responses
topP: 0.9,
topK: 45,
maxOutputTokens: 2048, // Allow longer output for SVG generation
stopSequences: []
},
// Safety settings
safetySettings: [
{
category: "HARM_CATEGORY_HATE_SPEECH",
threshold: "BLOCK_ONLY_HIGH"
},
{
category: "HARM_CATEGORY_SEXUALLY_EXPLICIT",
threshold: "BLOCK_ONLY_HIGH"
},
{
category: "HARM_CATEGORY_DANGEROUS_CONTENT",
threshold: "BLOCK_ONLY_HIGH"
},
{
category: "HARM_CATEGORY_HARASSMENT",
threshold: "BLOCK_ONLY_HIGH"
}
]
};
try {
// Execute API request
const response = await fetch(endpoint, {
method: "POST",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify(requestBody)
});
const data = await response.json();
// Process response
if (data.candidates && data.candidates.length > 0) {
const svgContent = data.candidates[0].content.parts[0].text;
return svgContent;
} else {
throw new Error("Response from API is not valid: " + JSON.stringify(data));
}
} catch (error) {
console.error("Error occurred during request to Gemini API:", error);
throw error;
}
}
3-2. Haiku prompt¶
// Set API key (in actual use, retrieve it safely from environment variables, etc.)
const API_KEY = "YOUR_API_KEY";
const MODEL = "gemini-1.5-pro-latest";
// Request configuration
async function generateHaikuFromImage() {
// API endpoint
const endpoint = `https://generativelanguage.googleapis.com/v1beta/models/${MODEL}:generateContent?key=${API_KEY}`;
// Base64 encoded image data
const imageData = "";
// Build request body
const requestBody = {
contents: [
{
parts: [
{
text: "Please create a haiku based on this Zen garden (karesansui) image."
},
{
inlineData: {
mimeType: "image/jpeg",
data: imageData
}
}
]
}
],
systemInstruction: {
parts: [
{
text: `# Role
You are a master haiku poet with deep knowledge of Japanese aesthetics and Zen philosophy.
# Task
1. Carefully observe the provided image of a Zen rock garden (karesansui).
2. Create a Japanese haiku that captures the essence, visual elements, and atmosphere of the garden.
3. Translate the haiku into English following the translation guidelines below.
# Japanese Haiku Requirements
- Follow the traditional 5-7-5 syllable structure (5-7-5音).
- Include an appropriate seasonal reference (kigo/季語).
- Reflect visual elements of the rock garden (line patterns, sand textures, overall composition).
- Embody Zen spirit and Japanese aesthetics (wabi-sabi/わび・さび, yugen/幽玄).
# English Translation Guidelines
1. **Syllable Structure:**
- No need to maintain the strict 5-7-5 syllable pattern in English.
- Recommended options:
* 3-5-3 syllables (11 total)
* 2-3-2 syllables (7 total)
* Free form but maintain a three-line structure
2. **Seasonal References:**
- Preserve the seasonal word (kigo) in a culturally appropriate way.
- Example: "名月" → "harvest moon" rather than literal "famous moon"
3. **Cutting Words:**
- Transform Japanese cutting words ("や"/"かな", etc.) into appropriate English punctuation (commas, dashes, exclamation marks) or line breaks.
4. **Translation Priorities:**
- Meaning: Clearly convey the original intent
- Conciseness: Avoid unnecessary words
- Imagery: Use vivid, evocative language
- Rhythm: Maintain natural flow in English
5. **Core Principle:**
- Translation is a cultural and poetic bridge, not word-for-word conversion.
- The spirit and beauty of the haiku should resonate with English readers as poetry.
# Output Format
You must strictly adhere to the following format and provide nothing else:
俳句: [JAPANESE HAIKU WITH 5-7-5 STRUCTURE]
英訳: [ENGLISH TRANSLATION FOLLOWING GUIDELINES]
Do not include any additional explanations, comments, or text outside this exact structure. Do not number the entries or add any formatting beyond what is specified.
# Examples
The following are examples of the required format:
俳句: 古池や 蛙飛び込む 水の音
英訳: An old silent pond... A frog jumps into the pond, splash! Silence again.
俳句: 桃の花 散りゆく様は 桜かな
英訳: The peach blossoms As they scatter Look like cherry blossoms.
俳句: 月一輪 星無数空 緑なり
英訳: Around the lone moon countless stars the sky now green.`
}
]
},
generationConfig: {
temperature: 0.85,
topP: 0.9,
topK: 45,
maxOutputTokens: 256,
stopSequences: []
},
safetySettings: [
{
category: "HARM_CATEGORY_HATE_SPEECH",
threshold: "BLOCK_ONLY_HIGH"
},
{
category: "HARM_CATEGORY_SEXUALLY_EXPLICIT",
threshold: "BLOCK_ONLY_HIGH"
},
{
category: "HARM_CATEGORY_DANGEROUS_CONTENT",
threshold: "BLOCK_ONLY_HIGH"
},
{
category: "HARM_CATEGORY_HARASSMENT",
threshold: "BLOCK_ONLY_HIGH"
}
]
};
try {
// Execute API request
const response = await fetch(endpoint, {
method: "POST",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify(requestBody)
});
const data = await response.json();
// Process response
if (data.candidates && data.candidates.length > 0) {
const generatedContent = data.candidates[0].content.parts[0].text;
return generatedContent;
} else {
throw new Error("Response from API is not valid: " + JSON.stringify(data));
}
} catch (error) {
console.error("Error occurred during request to Gemini API:", error);
throw error;
}
}
4. Program¶
https://github.com/tokuyama-y/mml
5. Website¶
https://machinical-memory-landscapes.cloud
6. Future improvements for the next spiral¶
- The current rake lift mechanism doesn’t support software-based Z-axis calibration. Replacing it with a pantograph or similar system would allow programmatic calibration and better adaptability to sand height changes.
- The rake’s passive rotation is limited by the short distance between its pivot and tip, causing it to lag behind movements. Extending the rake from the axis would improve responsiveness.
- The LED backlight isn’t program-controlled. Integrating it would enhance interactivity.