.Future Fruit.

Project Design and Development Blog A collaboration between Plant Developers and Interaction Designers. Creating Dynamic 3D Interfaces for Visualising and Experimenting with the the Future of Fruit Development. Supported by the Institute for Future Environments, QUT. Other Project and Teaching Blogs by Dr Deb Polson Colliding Dimensions | Abstractions | Design Fictions | Augmented Realities | ECOS Sim | Hybrid Cinema folio debpolson.com | twitter debpolson

[fruitfutures]

3D Printing Experiments

We can now export a STL file from the FF system ready for 3D Printing.

So far we have test printed a selection of custom designed fruits using a range of 3D printers at the QUT J Block Fabrication Lab... using a variety of plastic materials and different printing settings.

Next... we want to modify a printer to print edible gels!!

[fruitfutures]

World Science Festival 2017

Future Fruits was exhibited at the World Science Festival!!! http://www.worldsciencefestival.com.au/

“A celebration of science and art in Brisbane 22-26 March, 2017″. Bottom right photo from https://twitter.com/WSFBrisbane/status/845900652886872065 

It was such a thrill to finally see the public interact with our prototype (multi-touch version). The experience was very encouraging as it confirmed all of the design and technical decisions we made along the way... and also revealed some surprisingly simple ways to make some dramatic improvements.

Future Fruits was popular both with the kids and young adults as well as grown ups and visiting scientists. Each experiencing different aspects of the model depending on their interests and experiences. Watching first time users be delighted with a new project is so wonderful... but most rewarding is listening to how the project inspires people to discuss what this could mean about the future of food... priceless!

We observed people having a great time making new fruit... but also with experimenting with how the modeling system and user interface worked. As expected, the protrusion controls were the most popular.

A special thanks to the wonderful QUT student volunteers who represented us with contagious enthusiasm, boundless curiosity and pure smarts :-) 

[fruitfutures]

Custom Design Your Own Fruit

Here is the final version of the FUTURE FRUIT prototype... and a selection of my Future Fruits I designed on my Laptop today... in the future, we will be experimenting with exporting our custom fruit designs to a 3D printer with flavour and nutrients added!

We now have our FUTURE FRUIT interactive modeling system working beautifully on various interfaces/technologies including:

Desktop/Laptop machines

Multi-Touch Screens

Virtual Reality - HTC Vive

[fruitfutures]

Custom Designed Food

On Friday, November 3, Deb presented our final iteration of the FUTURE FRUIT prototype to the QUT community at IFE

In the new year, we will be looking for new opportunities to expand our design and science collaboration and hopefully continue to experiment and develop an more sophisticated demonstration of how we can imagine ways to engage in the customisation of 3D printed foods based on actual plant biology and with the flavours we want and nutrients we need...

If you are interested in our design process, science collaboration, research findings, or future plans... go to https://www.facebook.com/HubGames/ and leave a message for Deb.

[fruitfutures]

Main Development Tools and Interfaces

The HUB Studio team uses technologies normally used by game developers, to create interactive modeling systems that present scientific data and scenarios... for users to experiment with a range of variables and see the consequences of their decisions.

In the case of FUTURE FRUIT, we have created a 3D Interactive modeling system for the creation of new fruits based on fruit development biology.

Images (screenshots) from top left:

Unity 

Blender 

Shader Forge 

Unity Mesh generation - 

[fruitfutures]

Multi-touch Future Fruit Demo

In HUB Studio we currently have 2 x Multitaction MT550 55inch touch screens. These are the very same touch screens that make up the Cube at QUT.

[fruitfutures]

Virtual Reality Future Fruit Demo

We now have an early environment working in VR... HTC Vive!

Our ideal user experience would be:

Custom Design your new fruit

Add flavours and select nutrients

Send to 3D Printer

Enjoy the yumminess

[fruitfutures]

DesFi Prototypes - The Future of Food Production 

Producers of Science Fiction visualise the future.... Producers of Design Fiction create actual prototypes for the future - both explore possible future scenarios based on existing technologies, human interactions and imaginative predictions.... 

Future Fruits demonstrates two important future scenarios

3D Printing of food - with equal measures of nutrients and taste

Consumer food production - if you can print any shape, why print tomatoes?

This team has successfully prototyped a modelling tool that allows users to create their own fruits based on the familiar components of natural fruit development.

This prototype considers the future but tests it within the current limitations and opportunities... Our fruit simulator can now be operated on your computer, multitouch surfaces and in VR environments.

[fruitfutures]

Sprint Three - Environment Design

Exterior view - of our biolab we had decided at this point that the interior space was too vast... making the fruit model look too small and the space around a little redundant (however, we needed to reconsider this when we first entered the biolab in VR with our Vives)

Interior view - of our Future Fruit BioLab - a screenshot of a later version than the one above. This also shows the added biogardens that can be seen through the windows of the biolab. This suggests that this is where the future fruit models go for the next stage of ‘testing’?

[fruitfutures]

Sprint 3 Review

Sarah takes us through her latest renderings of our UI designs and

Wade’s spaceship get’s a test flight before it settles into orbit...

Ryan and Sarah go through the how graphical components can be prepared in illustrator for Unity integration

left image shows how different the latest build looks on the 55inch Multitaction and Deb’s 13inch Apple laptop

Sprint 4 Plan - Integrating all components into the latest build!

[fruitfutures]

Future Fruit Space Lab: Comparing Versions

Here is our initial environment design based on the premise of the Future Fruit experiments being housed in an imagined Future Lab.

Top image - Wade’s first draft of the Space Lab environment

Bottom left and right - version 2.0 and 3.0 of the working prototype (sliders and pickers on the left of both interfaces)

Version 3.0 shows both the impact of having a themed environment but also the impact of having more advanced fruit modeling controllers available.

Now the diversity of Fruit that can be visualised by this program is vast... it is expected that no 2 fruits will ever be exactly the same.

We have also added a ‘Randomiser’ option - users can select the Randomiser and it transitions through random designs. The user can select a random design to start from and make manual changes to it. A great option for interaction.

[fruitfutures]

Finalising User Controls and UI Components

Sprint Three Team Review

Great progress on the ‘External Mode’ !!!

We have added a sophisticated ‘Protrusions’ controller (see example top left image). These Protrusions have shape, count, strength, position and gravity properties.

Today we detailed every last UI component... all according to the fruit assets and formation behaviors:

Controller functions, properties and values 

Controller Categories for UI layout consideration (basic, advanced and finish)

Controller styles and mechanics (horizontal and vertical sliders, dials, and pickers). With increasing complexity across the categories as the fruit forms 

Camera positions for optimising view according to Controller selection (snap to side view height and weighting, top view for segments and 45 degree for top and bottom dimples)

Final UI Components and Categories 

Fruit Basic Shape - snap to side view

Height - verticle slider

Weighting (top bottom) - verticle slider

Width - horizontal slider

Top Dimple - snap to 45 degree

contrast

shape

Bottom Dimple - snap to 45 degree

contrast

shape

Curve (bend)

Advanced - snap to top down view

Segments - horizontal sliders

count

shape

Protrusions

count

shape

strength

position - verticle ‘dual’ sliders

ends

fall-off

Gravity or Droop - verticle sliders

Finish

Colour

Pick

Shift

Texture

type

density

strength

...

[fruitfutures]

Showing Sarah the New Build

Fingers crossed that Ryan and Wade stop being so ridiculously awesome for a while so that Sarah can start working on the UI components without expecting more Control Functions and an even more ‘External Mode’ properties...

[fruitfutures]

Rohan and Tomasz visit the Studio!

Rohan works with Ryan to integrate his ‘likeness’ code that automatically labels a Fruit Model with ‘Apple’, ‘Pear’ or ‘Orange’ if the user’s fruit model settings resemble those of a common fruit.

 We discussed the possibility of adding a function that calculates percentages of ‘likeness’ and re-names the fruit with a combination such as, ‘Pepple’... or with more data on more fruits, a ‘Kiwange’... or our favourite ‘Banapsicum’.

Even more interesting, calculate when a Fruit Model bears little resemblance to any existing fruit and offer a ‘uniqueness score’. For example, if a user’s model only resembles any other fruit data be 20%, then the ‘Uniqueness Score’ is 80%...? 

Worth considering for the next round.

We also discussed the progress of the work done for the ‘Internal Mode’... without a resolved volumetric model, Tomasz has agreed to deliver ‘experiments’ only with documentation for future development/integration or the ‘Internal Mode’

This workshop summary

Rohan - delivered code for labelling 'likeness' to fruit models

Tomasz - 'Internal Mode' (volumetrics) will be delivered as 'Experiments' with Documentation for 'Future Implementation'

Ryan and Wade - demonstrated new features for the 'External Mode'

Deb and Sarah - finalising UI components/mechanics

Other stuff discussed for documenting as Future Implementations:

Originality scoring for fruit as an extension to Rohan's code

More complex 'Likeness' feedback to create 'New' (mashup) fruit names such as Banapsicum

Volumetrics generator UI scenarios

Decisions on actual Deliverables

Final Demo - Vertical Slice Prototype - External Mode with extra yumminess (new 'protrusion', ‘texture’ and ‘bend’ properties/functions)

Volumetric Experiments - plus documentation of implementation plans

Documentation - Prototype Framing Doc and Recommendations for Future Development

[fruitfutures]

External Mode Vertical Slice

Today the design team met to define what we can feasibly do to complete the prototype given the current limitations in 

the provision of data (scientific/plant info)

the volumetric system for the ‘Internal Mode’

time left before deadline (15 days of development team over 2 months)

After much consideration, the following recommendations are made for final delivery of the External Mode fully resolved with ...

stable and customisable modeling system (procedural fruit model)

a fully resolved GUI (controls, sliders, pickers)

rendered environment (3D environment for presenting the fruit and GUI)

statistical ‘likeness’ feedback for External Mode UI (Rohan's data)

opening and closing screens (intro and credits screens)

formatted to a 16:9 aspect ratio for desktop/laptop download/use and 18:8 Cube 55″ Multitouch display

conversion model to format internal renderer (not integrated into the deliverable - for future use instead)

Images above: 

Current Screen Shot of the July Build currently under development with new controls and shader effects

Deb's design sketches and Ryan’s diagrams for UI prep

Deb (invisible) Ryan, Sarah and Wade discussing Sprint workflow for UI and environment asset development and integration

Although we had hoped to prototype both an External and Internal Mode, I am very pleased with our current plan ... to polish the External Mode to look and work in a sophisticated way will be valuable as a major delivery for the Catapult. This way we can all demo the prototype as a working 'vertical slice’ of a resolved sample section of the project that demonstrates both the mechanics and aesthetics.... 

... a beautiful, fancy, interactive, 3D Fruit Modeling System... developed in a sophisticated way that allows us to easily extend the model with further Modes of Interaction and Visualisation in the future.

[fruitfutures]

The Future of Future Fruits

We now have both the HTC Vive and Oculus Rift VR headsets in our CIP Studio.

This is a video of Ryan using the HTC Vive (headset) in the Valve Demo (The Lab) - we were especially looking for VR experiences that may help us imagine how we could design the Future Fruits VR environment and interactions... this demonstrates the 'slicing' of fruit to see the internal structures.

Very exciting

[fruitfutures]

The Boss Fight of Future Fruits

The Design and Development Team have been discussing what the ‘End Game’ for our Future Fruits prototype could be. An ‘End Game’ event is the final stage of some action or process... in the case of Chess, it is checkmate and for Super Mario it is a Boss Fight.

For Future Fruit we currently have the following modes of interaction, aka ‘levels of play’:

External Mode - modeling the shape, colour and texture of a future fruit

Internal Mode - designing the internal structure of this future fruit

Each mode has a set of inputs for user control (design) and outputs (system feedback on the fruit model)

Proposed next Modes:

Product Mode - would offer a number of 'product’ options for the fruit such as 'tastes  like' based on the prototype data (external/internal user decisions) and the system information (scientific data). this allows the user to imagine other qualities of a future fruit that the prototype does not yet model... or may never be able to? 

Market Mode - shows the user's fruit model in a category with other fruits... for example sitting in the 'citrus' section between a lemon and a mandarin...? 

Graphical User Interface Potential for modes

For Product Mode the GUI could be a drag and drop selection of ‘fruit qualities’ from a selection of ‘tastes’, ‘softness/hardness’ etc Presented in a similar way to ‘inventory’ options in a video game.

For the Market Mode the GUI could be something like what we currently have in Self Service Check Outs in supermarkets... but with a futuristic upgrade to match our current UI theme of Futuristic Biosphere...

So the 'End Game’ Scene for the Future Fruit user is a ‘Market Release’... with much fanfare! At Market Release (or some such name) the users ‘Fruit Dossier’ gets sent to a Community Market Display... an online display of designed Future Fruit by the user community... this is also the important data collected for Roger... to indicate user preferences!!!! Boss Fight Success!!!

The ‘Fruit Dossiers’ would be something like a merger of a Botanical Diagram and a Spy File :-)

Feedback on these options would be great... and will be on the agenda for todays team catchup...