Mobile AR Experiences @sekont-blog - Tumblr Blog

Apple adding augmented reality to iOS (IPhone & IPad) maps

A recent patent submission by Apple describes a technique to improve iOS (IPhone & iPad) maps using augmented reality. It proposes to integrate additional information by overlaying a camera video stream with supplemental information about the surrounding environment.

This approach seems to be very close to techniques used already by Layar or Wikitude. I'm really curious how Apple will implement this approach in a final product. Yet it remains unclear where in which way this technique is innovative!

"According to Apple's patent application, street names, addresses and landmarks can all be added to a scene simply by pointing the camera at an area in the vicinity."

Patents Show Major Changes Coming to Apple iOS Maps | The iPhone FAQ

#ios #iphone #iPad #AR #mobile

My take on WebGL

Hi there,

in recent news about WebGL, where Microsoft addressed concerns about browser security in its blog post WebGL considered harmful. Basically, they say that due to several reasons, like the inevitable access of WebGL code to the hardware (the graphics card), buggy drivers and other issues, WebGL represents a significant risk to browser security. Therefore, the MS Internet Explorer will not support WebGL in the foreseeable future.

Who doesn't know WebGL yet, "WebGL is a cross-platform, royalty-free web standard for a low-level 3D graphics API based on OpenGL ES 2.0, exposed through the HTML5 Canvas element as Document Object Model interfaces. Developers familiar with OpenGL ES 2.0 will recognize WebGL as a Shader-based API using GLSL, with constructs that are semantically similar to those of the underlying OpenGL ES 2.0 API. ", which is the definition of WebGL by the Khronos Group.

Of course, that created a some turmoil in the WebGL community and Gregg Tavares from the Google Chrome Dev team came up with a more than clear blog response. Basically, he is saying that Microsoft is all wrong. Every mentioned security issue can be adressed so that browsers can implement WebGL in a secure way.

In any way, Google will continue to push forward Chromes WebGL development (Chrome experiments), as Firefox will do (Mozilla Demos). I predict that MS will finally align, when it will become inevitable for them. This point could be reached, when popular games will be published using WebGL technology. You could play as you go in your browser, no install, no plugin (Flash). But thats still a long way to go.

Since I work for more then 10 years with OpenGL, of course I would like to see WebGL succeed. There are significant challenges on the way, but I clearly follow the position of Gregg Tavares that these problems can be tackled, solutions already exist or will be developed in the near future.

If you are new to WebGL, here is a small introduction and if you want to have more details on the topic do not hesitate to take a look at the Google IO WebGL Techniques And Performance presentation.

Of course, I'm especially interested in WebGL since this technology will probably allow mobile AR developers, in the not so far future, to create applications that won't be dependent on many different plugins or special applications for each mobile plattform.

You can get a good impression what I'm talking when you read previous blogpost on Remixing Reality by Mozilla presenting an AR application written in webGl and flartoolkit (flash) that even runs on Android Firefox beta.

I just played around with some WebGL and came up with the example below my other blog. Regards to learningwebgl.com. If you do not see a turning triangle than try another browser like Chrome or Firefox.

<![CDATA[// <![CDATA[ // <![CDATA[ #ifdef GL_ES precision highp float; #endif varying vec4 vColor; void main(void) { gl_FragColor = vColor; } // ]]<![CDATA[> // ]]]]><![CDATA[>]]><![CDATA[// <![CDATA[ // <![CDATA[ attribute vec3 aVertexPosition; attribute vec4 aVertexColor; uniform mat4 uMVMatrix; uniform mat4 uPMatrix; varying vec4 vColor; void main(void) { gl_Position = uPMatrix * uMVMatrix * vec4(aVertexPosition, 1.0); vColor = aVertexColor; } // ]]<![CDATA[> // ]]]]><![CDATA[>]]><![CDATA[// <![CDATA[ // <![CDATA[ var gl; function initGL(canvas) { try { gl = canvas.getContext("experimental-webgl"); gl.viewportWidth = canvas.width; gl.viewportHeight = canvas.height; } catch (e) { } if (!gl) { alert("Could not initialise WebGL, sorry :-("); } } function getShader(gl, id) { var shaderScript = document.getElementById(id); if (!shaderScript) { return null; } var str = ""; var k = shaderScript.firstChild; while (k) { if (k.nodeType == 3) { str += k.textContent; } k = k.nextSibling; } var shader; if (shaderScript.type == "x-shader/x-fragment") { shader = gl.createShader(gl.FRAGMENT_SHADER); } else if (shaderScript.type == "x-shader/x-vertex") { shader = gl.createShader(gl.VERTEX_SHADER); } else { return null; } gl.shaderSource(shader, str); gl.compileShader(shader); if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) { alert(gl.getShaderInfoLog(shader)); return null; } return shader; } var shaderProgram; function initShaders() { var fragmentShader = getShader(gl, "shader-fs"); var vertexShader = getShader(gl, "shader-vs"); shaderProgram = gl.createProgram(); gl.attachShader(shaderProgram, vertexShader); gl.attachShader(shaderProgram, fragmentShader); gl.linkProgram(shaderProgram); if (!gl.getProgramParameter(shaderProgram, gl.LINK_STATUS)) { alert("Could not initialise shaders"); } gl.useProgram(shaderProgram); shaderProgram.vertexPositionAttribute = gl.getAttribLocation(shaderProgram, "aVertexPosition"); gl.enableVertexAttribArray(shaderProgram.vertexPositionAttribute); shaderProgram.vertexColorAttribute = gl.getAttribLocation(shaderProgram, "aVertexColor"); gl.enableVertexAttribArray(shaderProgram.vertexColorAttribute); shaderProgram.pMatrixUniform = gl.getUniformLocation(shaderProgram, "uPMatrix"); shaderProgram.mvMatrixUniform = gl.getUniformLocation(shaderProgram, "uMVMatrix"); } var mvMatrix = mat4.create(); var mvMatrixStack = []; var pMatrix = mat4.create(); function mvPushMatrix() { var copy = mat4.create(); mat4.set(mvMatrix, copy); mvMatrixStack.push(copy); } function mvPopMatrix() { if (mvMatrixStack.length == 0) { throw "Invalid popMatrix!"; } mvMatrix = mvMatrixStack.pop(); } function setMatrixUniforms() { gl.uniformMatrix4fv(shaderProgram.pMatrixUniform, false, pMatrix); gl.uniformMatrix4fv(shaderProgram.mvMatrixUniform, false, mvMatrix); } function degToRad(degrees) { return degrees * Math.PI / 180; } var triangleVertexPositionBuffer; var triangleVertexColorBuffer; var squareVertexPositionBuffer; var squareVertexColorBuffer; function initBuffers() { triangleVertexPositionBuffer = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, triangleVertexPositionBuffer); var vertices = [ 0.0, 1.0, 0.0, -1.0, -1.0, 0.0, 1.0, -1.0, 0.0 ]; gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW); triangleVertexPositionBuffer.itemSize = 3; triangleVertexPositionBuffer.numItems = 3; triangleVertexColorBuffer = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, triangleVertexColorBuffer); var colors = [ 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 1.0, ]; gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(colors), gl.STATIC_DRAW); triangleVertexColorBuffer.itemSize = 4; triangleVertexColorBuffer.numItems = 3; squareVertexPositionBuffer = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, squareVertexPositionBuffer); vertices = [ 1.0, 1.0, 0.0, -1.0, 1.0, 0.0, 1.0, -1.0, 0.0, -1.0, -1.0, 0.0 ]; gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW); squareVertexPositionBuffer.itemSize = 3; squareVertexPositionBuffer.numItems = 4; squareVertexColorBuffer = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, squareVertexColorBuffer); colors = [] for (var i=0; i < 4; i++) { colors = colors.concat([0.5, 0.5, 1.0, 1.0]); } gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(colors), gl.STATIC_DRAW); squareVertexColorBuffer.itemSize = 4; squareVertexColorBuffer.numItems = 4; } var rTri = 0; var rSquare = 0; function drawScene() { gl.viewport(0, 0, gl.viewportWidth, gl.viewportHeight); gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); mat4.perspective(75, gl.viewportWidth / gl.viewportHeight, 0.1, 100.0, pMatrix); mat4.identity(mvMatrix); mat4.translate(mvMatrix, [-1.5, 0.0, -3.0]); mvPushMatrix(); mat4.rotate(mvMatrix, degToRad(rTri), [0, 1, 0]); gl.bindBuffer(gl.ARRAY_BUFFER, triangleVertexPositionBuffer); gl.vertexAttribPointer(shaderProgram.vertexPositionAttribute, triangleVertexPositionBuffer.itemSize, gl.FLOAT, false, 0, 0); gl.bindBuffer(gl.ARRAY_BUFFER, triangleVertexColorBuffer); gl.vertexAttribPointer(shaderProgram.vertexColorAttribute, triangleVertexColorBuffer.itemSize, gl.FLOAT, false, 0, 0); setMatrixUniforms(); gl.drawArrays(gl.TRIANGLES, 0, triangleVertexPositionBuffer.numItems); mvPopMatrix(); mat4.translate(mvMatrix, [3.0, 0.0, 0.0]); mvPushMatrix(); mat4.rotate(mvMatrix, degToRad(rSquare), [1, 0, 0]); gl.bindBuffer(gl.ARRAY_BUFFER, squareVertexPositionBuffer); gl.vertexAttribPointer(shaderProgram.vertexPositionAttribute, squareVertexPositionBuffer.itemSize, gl.FLOAT, false, 0, 0); gl.bindBuffer(gl.ARRAY_BUFFER, squareVertexColorBuffer); gl.vertexAttribPointer(shaderProgram.vertexColorAttribute, squareVertexColorBuffer.itemSize, gl.FLOAT, false, 0, 0); setMatrixUniforms(); gl.drawArrays(gl.TRIANGLE_STRIP, 0, squareVertexPositionBuffer.numItems); mvPopMatrix(); } var lastTime = 0; function animate() { var timeNow = new Date().getTime(); if (lastTime != 0) { var elapsed = timeNow - lastTime; rTri += (90 * elapsed) / 1000.0; rSquare += (75 * elapsed) / 1000.0; } lastTime = timeNow; } function tick() { requestAnimFrame(tick); drawScene(); animate(); } function webGLStart() { var canvas = document.getElementById("webgl"); initGL(canvas); initShaders() initBuffers(); gl.clearColor(0.0, 0.0, 0.0, 1.0); gl.enable(gl.DEPTH_TEST); tick(); } setTimeout(webGLStart, 100); // ]]<![CDATA[> // ]]]]><![CDATA[>]]>

#WebGL

The CARE project

"The CARE project proposes to apply Augmented Reality techniques to the cultural domain.

CARE is a French research project funded by the French research agency (ANR). The project brings together 7 partners, whose goal is to integrate, adapt and develop tools to enhance interactive systems devoted to cultural experiences. These enhancements are based on Augmented Reality and Emotions.

Besides the enhancement of the two specific scenarios considered in the project (Ballet of Biarritz and Museum of Toulouse), CARE results will include improved conception methods as well as new generic interaction techniques based on augmented reality technologies."

#AR #Immersion #Bordeaux

Reality Fighters for PlayStation Vita at GamesCom

"Fight as yourself anywhere and any time" is the motto of Reality Fighters for PS Vita give its debut at GamesCom in Cologne Germany these days. You can texture your fighter with your photo and play on real (back)grounds of your choice. Tracking seems to be relatively robust, for a planar surface of course, with sufficient lighting conditions. The projected shadows on the detected surface add very much value to the entire AR experience

#AR #PlayStation #GamesCom

Google acquires Motorola Mobile (and Spock is not impressed)

Google acquires Motorola Mobility (and Spock is not impressed) from NC

Only indirectly related to mobile augmented reality but still massive news. Hope this helps to drive innovation in the mobile sector.

More details on Googles Blog.

#Google #Motorola #Spock

KinectFusion: Real-Time Dynamic 3D Surface Reconstruction and Interaction - Microsoft Research

KinectFusion: Real-Time Dynamic 3D Surface Reconstruction and Interaction - Microsoft Research presented at SIGGRAPH 2011 in Vancouver Canada.

KinectFusion, a system that takes live depth data from a moving Kinect camera and in real-time creates high-quality, geometrically accurate, 3D models. Our system allows a user holding a Kinect camera to move quickly within any indoor space, and rapidly scan and create a fused 3D model of the whole room and its contents within seconds. Even small motions, caused for example by camera shake, lead to new viewpoints of the scene and thus refinements of the 3D model, similar to the effect of image superresolution. As the camera is moved closer to objects in the scene more detail can be added to the acquired 3D model.

#Kinect #Microsoft #SIGGRAPH #3D #reconstruction

Remixing Reality - Mozillas WebGL Augmented Reality Demo Rocks

Remixing Reality is a WebGL Demo by by ILMARIHEI published at Mozilla Labs that uses, as far as I can tell, Flashs Flartoolkit to create an amazing AR experience inside a website.

Flash with Flartoolkit allows you to integrate a prerecorded video presenting special markers. Flartoolkit computes then 3D positions of these 2D markers inside the video. These information are used by WebGL to render a 3D object or animation at the markers 3D position. Or in between two markers, in case of the jumping basketball animation.

In the next step they should get rid of Flash and Flartoolkit, so that no plugins are needed anymore to render the website. But this will take again a lot of time until some kind of augmented reality web standard will be established and most of all implemented.

Actually, I'm not 100% sure if they use Flartoolkit, but if you disable Flash the demo doesn't work anymore. So they use Flash at least to render the video. I suppose the tracking is done with Flartoolkit.

I just borrowed their demo ;). But only because I think its so totally cool and well, because it's so easy. If you like the demo, you should go to the Mozilla Developer site and share it on FB and Twitter with your friends. Switch to Chrome or Firefox if you cannot see a jumping basketball between two moving sheets of papers.

<![CDATA[// <![CDATA[ // <![CDATA[ // <![CDATA[ // <![CDATA[ // <![CDATA[ // Adaptor for demo protocol to make WoW demos run var ifrm = document.getElementById('demoframe'); ifrm.addEventListener('load', function (ev) { if (ifrm.contentWindow && ifrm.contentWindow.postMessage) { ifrm.contentWindow.postMessage('start_demo', '*'); } }, true); // ]]<![CDATA[> // ]]<![CDATA[> // ]]<![CDATA[> // ]]<![CDATA[> // ]]]]><![CDATA[>]]><![CDATA[// <![CDATA[ // <![CDATA[ // <![CDATA[ // <![CDATA[ // <![CDATA[ $('#share-opts li a').click(function () { var link = $(this); window.open(link.attr('href'), 'sharer', 'toolbar=0, status=0, resizable=1, width=626, height=436'); return false; }); // ]]<![CDATA[> // ]]<![CDATA[> // ]]<![CDATA[> // ]]<![CDATA[> // ]]]]><![CDATA[>]]><![CDATA[// <![CDATA[ // <![CDATA[ // <![CDATA[ // <![CDATA[ // <![CDATA[ window.handleIFrameEvent = function (name, data) { switch (name) { case 'liked': $('#demo-unlike').show(); $('#demo-like').hide(); break; case 'unliked': $('#demo-like').show(); $('#demo-unlike').hide(); break; } }; // ]]<![CDATA[> // ]]<![CDATA[> // ]]<![CDATA[> // ]]<![CDATA[> // ]]]]><![CDATA[>]]>

#AR #WebGL #Flartoolkit #Mozilla #3D

Unity Track Day at GDC Europe in Cologne

As part of European Game Developer Conference (GDC) in Cologne Germany, Unity3D will present a full day tutorial known as Unity Track Day. The different talks will give details about cross platform games, performance optimization and better social integration. Registration is already closed, but tickets are still available on site. More information here.

Here is the detailed program.

Hand’s demonstration on making cross platform games with Unity, Part 1Time: 9:30am – 10:20amSpeaker: Carl CallewaertA hands-on demonstration of the development of a multiplatform game starting with prototyping the game to the final version running on Android, iOS, the web, Facebook and as a standalone app. The steps include but are not limited to: programming, art assets import and updating, particles, GUI, AI and building for multiplatform deployment.

Hand’s demonstration on making cross platform games with Unity: Part 2Time: 10:50am – 11:40amSpeaker: Carl Callewaert

Performance optimization tips and tricks for UnityTime: 12:40pm – 1:30pmSpeaker: Jonas EchterhoffThis session will demonstrate optimization techniques on a sample Unity project, show how to use the Unity profiler to find performance bottlenecks, and will show some common pitfalls to avoid for optimal performance in Unity projects.

Shader Authoring in UnityTime: 2:00pm – 2:50pmSpeaker: Shawn WhiteIt’s difficult to make a game today without writing at least one shader. Whether you’re indie, mainstream, or somewhere in the middle, having the ability to write your own shaders will give you what you need to make your game POP! Unity provides a suite of built-in shaders to get you started, but in this talk we’ll get you up to speed on what you need to know about writing your very own shaders in Unity.

Being Social: How Games Can Leverage Facebook on the Road to SuccessTime: 3:20pm – 4:10pmSpeaker: Tom HigginsThe explosive growth of social networking is changing the ways in which people of every type interact with each other in some incredible ways. That’s easy to say but it takes a bit more effort to properly understand. What are those changes? Are they limited to those creating browser-based content or do they extend to the desktop and mobile apps as well? And most importantly, how can you and your team take advantage of them in your games? This presentation will discuss some of the most prominent questions facing developers targeting social networking sites and describe how developers can put that information to use right away. The discussion will focus on Facebook and the Unity game engine in particular as they offer developers unique opportunities for success. But in the end the strategies that will be shared can easily be applied anywhere, regardless of what social networking site or game engine is being used. Developers of all types can attend this session and learn how to incorporate social networking features into their own efforts, from the early design phases all the way through to release, giving themselves a much greater chance of success.

Using and publishing content and extensions on the Unity Asset StoreTime: 4:40pm – 5:30pmSpeaker: Tom HigginsThis session will start with an introduction to the Asset Store, showing what is available and demonstrate how to search for and use content in your own projects. After that he will go through how to submit your own content and extensions to the store, how it should be formatted and go through common mistakes to avoid and other hints for speeding up the submission process.

Distribution with UnionTime: 5:40pm – 6:30pmSpeaker: Brian BruningThere’s never been a better time to be a game developer. Huge audiences are now reachable without the aid of publishers or retail outlets. But what are the odds of getting noticed out there today in the new digital landscape? What tactics are most effective? What should you be doing to get noticed and find customers?

#Unity3D #Conference #GDC

Talk on mobile Augmented Reality at Paris Android User Group (Paug)

Hi there,

here is the video of our mobile augmented reality talk given at Paris Android User Group in june.

In the first part Atef Haouari presented a technique to create a mobile AR application for Android using position (GPS) and orientation sensors.

There is a recap of the entire talk here. In the second part (from 1:07h), I present some of our work in the area of vision based mobile augmented reality for Android using the Qualcomm AR SDK (QCAR) and Unity3D.

The presentation was held in french. Sorry for all the non-french speakers.

Watch live streaming video from parisandroidug at livestream.com

#AR #mobile #QCAR #paug #android #Unity3D

The Augmented City

Metaio just released a new video of their recent The Augmented City demo. Looks like they finally implemented a quite stable markerless visual tracking (SLAM??). The 3D rendering and animation is very fluid and appealing as well. Will be interesting what else they will announce on their upcoming insideAR conference in september in Munich.

#AR #mobile #Metaio #3D

Autodesk released new Sculpting 123D app for IPad

The app allows you to do some nice 3D modeling with your fingers. It proposes multitouch interfaces for moving, rotating and scaling. Another nice element is how they allow the user to color or texture the object. Rubbing on an image which seems to be projected on the geometry. You can even take a picture and use it for texturing.

Give it a try. Its free for a limited time. Available on the appstore.

#iPad #3D #sculpting #multitouch

AR Game using SLAM on IPad2

13th Lab recently released their new AR shooter Ball Invasion for IPad2. It uses "NASA" technology to track the environment and integrate the virtual objects you can shoot down.

Basically, they implemented a mobile version of SLAM that works pretty well in front of planar textured surfaces, like a wall for example. The shooter is fairly simple. The concept is not really new or innovative, neither are the graphics, but they got a point to have brought SLAM technology to the IPad2. It works quite stable though.

#AR #games #ios #mobile

The Augmented Reality Korean Unification Project

"The Augmented Reality Korean Unification Project unites North and South Korea with augmented reality [AR]. The project tries to heal the scars left by years of conflict in the Korean peninsula by removing the Korean Demilitarized Zone [DMZ] and returning it to its natural state before Korea was divided. The public may view Korea as a unified country as it once was. The Korean Unification Project removes weapons, checkpoints, fortifications, barriers, walls, and all reminders of the ongoing conflict from the Korean landscape."

Link The Augmented Reality Korean Unification Project

#AR #politics

SIGGRAPH Emerging Technologies Lab

Hi there,

its SIGGRAPH time. This year in wonderful Vancouver. Unfortunately I cannot be there, but there are some great new technologies to gaze at, to test and to touch. So if you are there, enjoy and share your experiences with us! Or at least with me.

A venue, often overseen at SIGGRAPH, is the Emerging Technologies Lab (short ETech). Although it is probably the most spectacular place at the conference, where you can see the most recent emerging interactive and graphics technologies. This can be new interfaces, (3D) displays, input devices, robotics, sensors, etc... Unfortunately, there is no augmented reality demo this year. Watch the video preview below to get an idea about ETech.

In the following I would like to shed some light on several selected projects, that seems to be particularly interesting.

Toucheo - The Immersive Multitouch Workspace

"We propose a new system that efficiently combines direct multitouch interaction and 3D stereoscopic visualization. In our approach, users interact by way of simple 2D gestures on a monoscopic touchscreen, while visualizing occlusion-free 3D stereoscopic objects floating above the surface at an optically correct distance."

ISCAN3D and IFACE3D: 3D Scanners on iPhone by Digiteyezer

"The first 3D scanner that works on an iPhone and cloud computing architecture, thus offering ease of use and oustanding ROI"

Surround Haptics: Sending Shivers Down Your Spine

"Surround Haptics is a new tactile technology that allows generation of high-resolution, continuous, moving tactile sensations anywhere on the human body."

Touch Interface on Back of the Hand

"This new computer-human interface uses the back of the hand for inputting data."

Face-To-Avatar

"This floating avatar system integrates a blimp with a virtual avatar to create a unique system of telepresence. The blimp avatar contains several pieces of equipment, including a projector and a speaker as the output functions."

True 3D Display

"This research team was the first to use laser-plasma technology for a true-3D display device that allows users to draw 3D images in midair [..]. Now the team has developed a much more compact and precise display [..], based on advanced laser technology. One remarkable feature of the new device is its enhanced resolution: from 300 points per second to 50,000 points per second. It displays 3D objects more faithfully in real time and increases the range of possible applications."

Hope you enjoyed this small survey and if you are in Vancouver don't miss ETech.

#conference #technology

Qualcomm AR SDK for iOS (IPhone & IPad)

Hi there,

just got my hands on the new Qualcomm AR SDK for IPhone and IPad, which you can download here. I tested the plugin Unity3D 3.4 and it works as smooth and easy as the earlier released Android SDK . There is also an SDK for XCode available, but I didn't test it yet. It's just so much easier to do some initial prototyping with Unity.

On a first glance it is very much the same as the Android SDK. There is the ARCamera object and the image targets. So if you familiar with the Android release you could probably port your project very fast to iOS. Which is great news.

An initial test (image right) using the standard Teapot example with two markers runs perfectly on IPad2. I did not measure the framerate for the moment, but I think it is far from touching the IPad2 resource capacities.

To get started Download the QCAR Unity plugin for Unity3D (you need the most recent Unity3d 3.4!). Unzip, install the included app and it will extract multiple Unity package files among you will find QCAR-ImageTargets.unitypackage. Load (just double click) this package into Unity3d and choose the delivered test scene. It contains two targets, each visualizing a classic Teapot. Deploy for iOS. Unity3D will create now the corresponding XCode project, which you can compile in XCode and deploy on your IPhone or IPad.

Have fun

#AR #QCAR #IPad #IPhone #iOS #Unity3D

Plugins for Unity3D

Hi there,

during the following weeks I will write a small series of posts about plugins for Unity3D. In particular how to create plugins for Android and Windows applications.

While Unity3D already comes with a big amount of features, not all of them are available on all platforms. Especially the mobile platforms lack of many functionalities. Using plugins, you can use additional features in your project. You can integrate your own sound plugin, give access to a particular render engine or use other external APIs. Plugins can also be very useful if you already possess a library with functionalities that you are interested in using.

Unity3D allows you to write your individual plugin, which you can access through interfaces (C# or Javascript) inside your project. The integration of your own plugin is not necessarily difficult but can easily become cumbersome, if you are not familiar with concepts, like Dllimport or JNI.

The basic structure of a Unity3D plugin consists of multiple layers. At Unity level a Javascript/C# interface layer, a layer that serves as a wrapper (like JNI), and finally your_library layer (C/C++/Java). The underlying concept of such an approach is to define interfaces (interface layer) that will give you inside Unity access to methods implemented outside of Unity, for example in a particular library (your_library Layer). Such access is often realized using a wrapper, like JNI (wrapper layer). JNI (Java Native Interface) is a popular tool to access native (C/C++) code from Java, which you might know if have already worked with the Android NDK (Native Development Kit).

In the following, I present an overview of the some common methods to access external functionalities on the Android and the Windows platform.

Android

A very easy way to access a particular Android function from Unity3D, without the need to implement a particular plugin, is the AndroidJNI interface provided by Unity3D. The wrapping is done internally by Unity3D, so you do not have to worry about that. The drawback is that you can only access functions provided by the Android SDK Unity3D is linked to.

Accessing Android from Unity3D via AndroidJNI interface

There are basically two ways to implement Unity plugins for Android applications. Firstly, using Java and the Android SDK or secondly in C/C++ using the Android NDK, as illustrated in the following illustration.

Accessing Android from Unity3D using JNI

Windows

Plugins for Windows applications are implemented using a so called Dllimport, that gives access to an external shared library (dll) under Windows.

Access to an external dll library

In the several follow up posts I will go in more detail on each of these approaches.

Links

Unity3D Plugins Documentation

#Unity3D #Android #Plugin

Breizhcamp recap

Hi there,

Last friday I went to Rennes to give a talk at the Breizhcamp on mobile augmented reality for Android using the Qualcomm AR SDK and Unity3D. Bassically an extended version of my previous PAUG talk in Paris. Slides are below. There will be as well a video I will post as soon as I get my hands on it.

Anyway, Breizhcamp was really cool. A lot of nice presentations around GWT, .Net, Android, iOS etc. There was also a talk about Kinect, which was interesting and quite entertaining at the same time.

Breizhcamp Rennes 2011 View more presentations from sekond0.

Trending Blogs

Recently Viewed Blogs

Mobile AR Experiences