Zihui-Gavin @zihui-gavin - Tumblr Blog

08 Some combinatorial logic thinking

07 head translation

Each head movement of the dancer is extracted as an influence point in space—the height of the head determines the degree of local roof bulge, the position of the head in the plane determines the location of roof undulations; the distance between the shoulders determines the size of the influence area—the roof coverage increases when the shoulders are extended, and the roof tightens when the shoulders are retracted; the moment of highest music intensity is identified as the rhythm peak, generating skylights or openings at the corresponding locations. All influence points are superimposed and merged to form a continuous roof field, and a Roof Canopy with thickness is generated through smoothing.

head— Core Data Source The planar position (XZ) of the head determines the location of roof undulations; the vertical height (Y) of the head determines the degree of local roof elevation. Each head sampling point generates an influence field in space, and multiple influence fields are superimposed to form a continuous roof shape.

Shoulder— Controls Roof Coverage The distance between the left and right shoulders determines the radius of the head's influence field. When the shoulders are extended, the influence area increases, and the roof expands outwards; when the shoulders are retracted, the influence area decreases, and the roof becomes more concentrated.

Music— Controls Roof Openings

The highest audio intensity peaks are extracted and mapped to the corresponding body movement positions. Each peak will create a skylight or opening on the roof surface; the higher the music intensity, the larger the opening.

06 Foot joint translation

Ankle – The core data source. The vertical height (Y-value) of the ankle joint is used to detect the moment of foot landing, the horizontal position (XZ) determines the step's location in space, the movement speed determines the step width, and the movement direction determines the step's orientation.

Knee – Determines the force of the foot landing. A large knee bend indicates a forceful step, with hard edges used for transitions between steps; a straighter knee indicates a lighter movement, with gentle slopes used for connections between steps.

Hip – Assists in calculating knee flexion. The distance from the hip to the ankle joint is used as the baseline leg length, normalizing the knee flexion ratio.

Each moment of a dancer's foot landing is captured as a step – the height of the foot landing determines the step's elevation, the movement speed determines the step width, the direction of movement determines the step's orientation, and the alternation of left and right feet determines the step's lateral offset. All steps are arranged from low to high height and connected by curves to form a walkable dance landscape.

05 The attempt in the realm of music space

This component turns ears and music into acoustic panels, sound rings, lighting nodes, and rhythm breaks.

Ears 7/8 → left / right acoustic interface

Ears + hips → sound field center

Bass → thickness + weight stronger Bass = thicker panels, larger rings

Kick peaks → rhythm nodes strong kicks = spheres, pulse rings, local thickening

BPM / peak density → number of panels faster rhythm = denser repetition

Audio intensity → panel height

Breaks → gaps and silence zones

Top strips → light bands on acoustic fins

Sound Logic

Sound center → central disc + wave rings

Side ears → two curved panel fields

Bass → mass and depth

Kick → impact nodes

Break → interruption, gap, or missing panel

This is not the final form at present. Let's first try to convert the music into some less functional components in the space.

04 main space

The main space is generated from hip movement, not from a predefined room.

hips 23/24 → ground field

Basic Rules

hip midpoint → space center hip path → main axis hip distance → width hip height → raised / sunken ground hip rotation → twist Bass → thickness + weight

Lines vs Blocks

continuous motion → lines Lines show boundaries, contours, flow, and height change.

selected hip moments → blocks Blocks mark key body positions, rhythm accents, or local platforms.

Six Space Types

high hips → raised stage Upward convex ground + small platform pads.

low hips → sunken dancefloor Downward basin + contour edge lines.

high-low hip jumps → split-level terrace Stepped ground + level-change lines.

long hip path → linear gallery Stretched floor strip + sequential pads.

strong hip rotation → twisted forum Rotated field + surrounding blocks.

wide hip opening → expanded hall Broad open ground + wider floor plates.

Scene + Music

outdoor video → larger field indoor video → smaller field strong Bass → heavier edges + thicker blocks

03 Opening / Window / Sightline

This component turns looking, pointing, and hand opening into entrances, windows, frames, and view corridors.

Nose 0 + eyes → main viewing direction

Eye center 2/5 → window / frame center

Eye inner + outer points 1 /3 / 4/ 6 → opening proportion

Head turning speed → opening density fast head turn = smaller, fragmented openings slow head turn = stable, larger openings

Gaze dwell time → opening size longer gaze = larger frame

Distance between wrists 15/16 → doorway width

Index fingers 19/20 → sightline guides / narrow view slots

Opening Rules

Stable gaze → large clear opening

Fast scanning → multiple fragmented apertures

Wide hands → wider entrance

Pointing gesture → directional slit or view corridor

This component is not a combination. Its components are distributed to be used for windows and walls in the space.

02 Handrail

This component turns hand movement into touchable boundaries: handrails, low rails, posts, grips, and light nodes.

Wrist path 15/16 → main handrail curve

Wrist height → corrected handrail height raw wrist height is remapped to about 850-950mm

Elbow-to-wrist direction 13/14 → rail offset + turning points

Pinky 17/18 → outer boundary line

Hand spread → rail thickness

Thumb 21/22 → grip points / connector nodes

Wrist pauses → posts

Strong pauses → light nodes on top of posts

Simplification Rule

Raw hand tracking is not used directly.

Small jitter is removed.

Only pauses, turns, spread points, and key inflection points are kept.

This makes the rail readable as architecture, not motion noise.

01 wall

This component creates one main wall first. All other parts grow from that wall surface.

Shoulder width → passage width + wall scale

Shoulder rotation → wall bending / turning

Shoulder height difference → sloped wall top

Elbow offset 13/14 → surface relief blocks

Strong body turns → vertical ribs / creases

Shoulder tilt + elbow fold → top folded plates

Music Break → cracks, gaps, displaced wall pieces

How The Wall Details Appear

Large protruding blocks → elbow bends push the wall surface outward

Thin vertical strips → strongest shoulder-turn moments

Top horizontal plates → wall top folds where shoulder tilt and elbow motion meet

Cracks / splits → low-energy music breaks

Surface bumps → small changes in shoulder width, turn, and elbow offset

06 version 2

Through clustering, four fragments that were identified as most similar to the bedroom environment were selected for preliminary building experiments.

03

Each model is first extracted as a set of spatial features, including body proportions, movement trajectories, cloud-like encapsulation, clothing involvement, horizontal extension, and vertical concentration, etc. The SOM will place similar models in adjacent areas based on these features, and then cluster them into 8 spatial tendencies. Red and orange mainly represent path-like spaces, emphasizing movement trajectories, foot movements and continuous lines. Blue and yellow mainly represent cloud-like enveloping spaces, suitable for developing into performance, gathering or immersive sound spaces. Green, cyan and purple are more inclined towards layered mixed spaces, reflecting multi-layer superposition, participation of clothing volumes and composite site characteristics. White is less in quantity, but is more inclined towards compact node spaces, with the potential for vertical concentration and small enclosure.

02

This week, we updated the models for the hands and the foot trajectories, unified the parameters of all four types of models, and combined them. Now there are a total of 3,266 combinational sub-models.

05 version 1

Based on the spatial coordinates of the hip joints in the new dance video, a new trajectory is drawn in space, varying over time.

Then, the audio intensity is extracted from this new dance video, and the 10 moments with the highest intensity are selected to obtain 10 points. Fragments are then placed into these points in sequence.