divein coaching

Thoughts about immersive, data based coaching

[diveincoaching]

Lean course creation

Startups run lean, but education startups are fat. Here is what I mean: Lean startups quickly iterate in tiny cycles toward a minimum viable product that matches customers needs. They operate through market pull rather than pushing a product that nobody wants. Now take education startups: Courses are created based on what we think customers might want. If it fails, it will fail down the line, when costs have already been incurred.

So, is there an alternative way? What follows is a lean sequence:

Start from an idea based on qualitative interaction: Qualitative here is a fancy word for talking with your potential customers and checking what they want. Sure you can go fancier and run more formal focus groups and all, but the main thing is to have an idea of what they want to learn rather than what you want to teach.

Run a market research: Offer something they want in return for a couple minutes where they will listen to a brief description of the course you are proposing, and simply ask whether they would take it and how much they would pay for it. It has to be that simple, so that you can ask a large enough number of people

If the ROI is there, build a first chunk of your course: Don't waste time building something sophisticated if all they need is something simple. Also, don't build something that is missing something they want. How can you tell? Well, build a minimum viable chunk of the course and present it to them. Go qualitative, meaning, show it to one person, fix the problems, show to another, and so on, until you get something that achieves their objectives, whatever that might be

Finish the course and start experimenting: Now that you know what works, complete the course and start formally experimenting with larger numbers. By experimenting I mean flip a coin and randomize. Is a certain type of video better than another? Is an immersive environment better than a plain vanilla course? Don't argue about it, simply experiment

If your course is getting traction, further identify what specific subgroups might want and segment your course: Here you have conjoint analysis, pricing analytics, market segmentation to help you identify niches that would be best served by the course being taught in a certain way that specifically matches their needs.

[diveincoaching]

Expertise: from examples to schemata

What follows are just some thoughts coming out of a conversation with Joao Vissoci, largely unsubstantiated by any empirical research. With that out of the way, I personally think that the holy grail in the coaching of the next generation of innovators is how to get them from some a novice to an expert status.

Notice that when we talk about innovators, the terms novice and expert have a slightly different connotation: Novices are people with very initial and somewhat naive ideas that have not been tested before, very error prone, while experts in a field have already made enough mistakes to start getting things right within that area at a higher percent of the time. So, for example, I might want to develop a technology to help patients make the diagnosis of common medical conditions using expert advice only when needed, but in practice I might be overestimating people’s level of computer literacy. As a result, the technology will either sink or will have to be substantially modified in order to create something useful.

But then, how does a novice go from making a naive assumption about the building of a technology like in my example, all the way to catching the problem from the very beginning and gearing the project toward a more viable direction? Well, here is a possible model:

Individuals start with examples. So in our case, I might have either have tried to develop the failed technology myself or have heard or read about a failure. I am talking about failures since failures tend to generate more lessons than successes, but this is obviously also applicable to examples of successes.

From the example I then create some kind of rule involving a connection between two concepts. In the previous case the two concepts would be (a) technologies and (b) simple, and the connection could be something like “should be,” meaning that “technologies should be simple.” In other words, when similar examples show up that rule would be applied. Because you have two concepts connected by a statement, let us just call that a triple.

The next step is to start aggregating multiple triples into a single schema, a schema here meaning an aggregate of triples. So, the schema could contain anything going from a further specification of the previous triple such as “technologies should be simple when applied to the general population” or other rules still connected to one of the concepts such as “failed technologies mean company failure” or “simple is important for anything.” The former connects to the overall schema through “technologies” while the latter connects to the schema through “simple.”

The so called expertise happens then when a sufficiently large group of triples and corresponding schemata are formed, so that whenever a new situation arises one or more concepts from the problem will fire a connection with a schema. Of importance, the more connections a given concept might have in an experts schemata, the more likely it will be highlighted when a problem comes up. For example, if my expertise entirely relies on making things simple, whenever a problem is posed to me, even if outside of my realm of expertise, I will likely search for something requiring simplicity, since that will allow me to bring up my whole schemata.

The open questions then are how do you create systems with good user experience (UX) so that novices can quickly create and use schemata that are aligned with those of experts as well as have the ability to challenge different schemata to build something new that will allow them to try different approaches.

[diveincoaching]

Barnes & Noble, flies, and embeddedness

A couple days ago I was walking around a mall looking for a restroom, and the closest one available was the one at Barnes & Noble. I am a book addict, and although in the past Barnes & Noble had been a constant location, with ebooks coming along I had not been into the store for at least six months.

As I walked into the place, aside for some flies here and there, the place was really empty, nothing resembling what it once was. This is probably a sign that the future for brick and mortar bookstore chains is not all that bright, but this is no news. What stroke me however, was how the experience of walking close to the shelves made me discover a number of interesting books, seeing connections that I would not be able to see on the web. Of course I didn't buy anything since the price of an ebook is not only lower but also the book itself is much easier to carry, but nonetheless the question remained: Why is the experience different and, I would risk saying, richer?

At the risk of running into folk psychology, I think that one of the potential answers is that bookstores bring along the concept of spatial immersion, and with our currently technology this experience can hardly be reproduced on the Web. I am no specialist in spatial or embedded immersion, but in my personal experience I can detect at least the following aspects:

Books are organized in a certain order, and this order forges associations among different topics

Being immersed in a bookstore your visual space can literally go in every single direction, and this limitless vision abolishes the constraints we are so used to when it comes to small screens.

There are other emotional stimuli involved, such as smelling the paper all over the place, the smell of paper probably evoking memories of fun hours reading interesting novels or interacting with interesting information

There is certainly far more, these are just notes from a first impression, the main point that I am trying to get across here is that it is no wonder that online education has such huge attrition rates, as it is not stimulating as many senses as a regular environment does.

[diveincoaching]

Globally distributed teams and course preparation: Up to which point does being collocated matters?

Before I get started I should have a full disclosure in place: I have been working with globally distributed teams for as long as I can remember. And even when my team members were theoretically close to me, I kept insisting that they didn't have to come to our office as long as they got their tasks done. So, this is all to say that I don't have much of a comparison group. With that out of the way, here are some thoughts:

Personal motivators: Now I'm not the most social person in the world, but one of the nice aspects that comes with socialization is that you get to know what is truly important for other people. So, is it just the money? Despite all the economic theory, that's surprising the case. People are certainly looking forward to not having to think about money or being concerned about how they are going to pay for their bills, but when that is taken care of they are usually in search for something else. For some it could be the adventure, for others it could be doing good, yet for others it could be fame and prestige, often times a combination of all of the above. Although with collocation the ability to talk for longer periods of time comes more naturally, with video conferences such as Google Hangout or Webcam Toy I would argue that the opportunities for socialization are even greater. In other words, it is easier to spend more extended periods of time with other people while getting to know what is really important for them.

Global sources of expertise - One of the most appealing features of globally distributed teams for course preparation is the ability to tap into talent and expertise that would otherwise be nearly impossible to get at a local level. By that I mean psychometricians, user experience specialists, growth hackers, assembly and testing specialists, copy-editors, and obviously content generators and trainers. All of a sudden, anywhere in the world is fair game, and now you become a global head or gem hunter.

Time zones - Having different time zones are not impossible, but people have to be flexible. This means that the 9 to 5 is definitely dead, that going to sleep after midnight means that you can sleep in the morning, and that you have to have a very good joint calendar such as Google calendar and an international SMS system such as Whatsapp allowing you to get in touch with people any time of the day or night.

Having time to dedicate to the project - Now, this might seem like an obvious thing, but since course preparation might not be the primary activity for some of the team members, it is important that they have a commitment to dedicate enough time to the project. Of course, in order to get to this stage there are important factors such as aligning financial incentives and of course giving people the time to adjust their lives to the new commitment.

Having a more or less common cultural understanding - This is no news, but when working with a culturally diverse group there will be messages that simply will not get across. And so it is important to be patient and give people the benefit of the doubt. For example, in some cultures being wrong is a reason for losing face, in others it is a reason to laugh. In some cultures if somebody disagrees with you and that person is your superior you should be quiet and then continue with your opinion. In others you should discuss. Since our group is based is the US, I won't deny that the overall culture is American, but I will also say that there is nothing nicer than hearing about and acting using new standards. And this is not about being PC, it's just that it's so cool. Now, one of my tricks to circulate around different cultures is to ask questions during non-threatening times. So, for example, if during a situation where a problem was raised and then person who was supposed to answer simply stayed quiet, why did that happen? What was the meaning and what should my reaction have been? In other words, it's important to try your best to learn "the other way."

Language - for very practical reasons, English is the dominant language. I speak a couple other languages and using them has certainly opened a ton of doors, but in bigger meetings English is usually the default. The challenge arises when you have a staff member who is absolutely outstanding but doesn't speak English. The issue there is that this person will likely get undervalued by others primarily because the communication is not there. What I try to do, and it's not easy, is to constantly communicate to others the value this person is bringing to the company and to the specific project at hand. This is not easy though, and so the other advice is to provide every single possible incentive for that person to at least try to communicate in English. With enough time things tend to get far better.

Give more than you receive - I once read this one from Tim O'Reilly and although it might seem like an purely altruistic statement, it actually is a core business value. And I will go beyond: You have to have something unique to offer. In my case I guess it has traditionally been the academic component of whatever work I do. So, even if I am working on a for profit startup, I will naturally attract those with a thing for academic work since they know I can help them out in that area. This means that with that arrangement I have the ability to provide them with something that is of high value to them, and I can then benefit from all the value they bring to me and to the company. I guess that's called a win-win.

[diveincoaching]

Immersive skill training sounds great, but does it add real value?

  Whenever one talks about immersive skills training, the reaction is almost invariably positive. Sure, it should be better than simply trying to get people to absorb information. But is this true in all cases?

Well, it depends on a number of factors, like for example what do you mean by immersion. I have recently talked about that and so won’t go there again. Instead, this time I would like to defend the argument that immersion in skill training is only worth the effort if the execution of the final skill in the real world depends on something that happens beyond somebody’s brain. Now, that was a pretty loaded statement, and so let’s go for some definitions.

According to Situated Cognition, individuals have their cognitive skills and decision making influenced by a series of factors that are external to their brain. The traditional categories for when cognition happens or is influenced outside of somebody’s brain are constituted by the 4 Es: Extended, Embedded, Embodied, and Enacted.

Extended cognition happens when the skill is extended by other people or technologies. For example, my ability to be a leader is obviously influenced by the people I lead, and therefore my learning leading skills would benefit from being in a situated learning environment where I can interact with fictitious characters that would pose leadership problems and would also react to my leadership decisions. The same would be true if my learning a skill would depend of my choosing which tools to use in a certain environment. For example, if I am being trained as a data scientist, I can make use of a wide variety of data analysis tools to tackle a certain problem. My cognition is clearly extended by these tools, and so my training would benefit from a situated learning environment where I will have to choose which combinations of tools I should use for each individual problem.

Embodied cognition is related to emotions. For example, let’s say that the objective is to train a healthcare professional who has counseling to patients who are a high level of stress. The training of these professionals would benefit from an situated learning environment where emotions can be elicited, since the skills developed in this environment will already be adapted to the environment where they will later be used.

Enacted cognition relates to anything that is learned through repetitive practice, especially in the context of repetition with an increasing difficulty level. I posted before about this topic, but the main idea is that in immersive environment should present students with a level of difficulty that is always a bit beyond their current level, so that they can continue to be stimulated while always learning more and more.

Finally, embedded cognition relates to environmental factors. For example, imagine having to train somebody in relation to time management skills, and then teach that person to place stickers around the house or comments within documents she is working on, so that every time she comes to a certain place or reads a certain document she can be reminded about activities that she has to do. In other words, the environment surrounding a given task provides clues about what has to be done next.

So, that’s it, the proposal here is that situated learning will have its place whenever the skill to be learned is substantially influenced by any of the four Es, which includes other people or technologies (extended), emotions (embodied), requiring repetition or practice (enacted), or depending on the environment (embedded).

[diveincoaching]

Immersive training for skill effectiveness - what are the secret ingredients?

In a previous post I talked about the idea that immersion in skill training can refer to both fun immersion or situational immersion. The latter, I argued, is related to being trained in a skill that actually means something in relation to an environment where you will be using that skill.

That's all good and well, but how do you actually go from theory to the practice of creating an immersive situational environment? Anyone, claiming to have definitive answers would probably be either stupid or insane, but here are some guesses:

Specialists are required: Creating situated immersion is close to impossible unless the person designing the immersive training environment is a specialist. The reason is that there are so many details in how that specific skill is used in practice, that simply getting a generalist would not get the environment to reach a level with enough quality.

Specialists don't necessarily know what they know: The idea is not new, and so let's start with a classical example on something that you might be an expert at: Riding a bike. Now imagine that I introduce to little Chloe, a four-year old who would like to learn from you how to ride a bike. To make it easier for Chloe, I now ask you to write a one-page document explaining how you ride a bike. And please write down specially how you get to balance yourself on that two-wheel thing without falling to one side. Not easy, right? Well, that's what in the psychology literature is known as tacit knowledge, basically meaning that experts usually don't consciously know all they know. Good news is that there are methods to extract that info from them, at least a part of it. I will talk about a couple principles, but if you want to know more I would recommend the outstanding book by Crandall on Cognitive Task Analysis or the science/art of mapping how experts think.

Tacit knowledge elicitation starts with specialists thinking aloud: So, you are now trying to create an immersive situational course and you have a specialist who is fully committed. But now you also realize that she doesn't know all that she knows. A very good start is to use what is called a "think aloud protocol." To make a long story short, present a specialist with a situated problem, or a problem in a situation that she is likely to find in daily practice, and then ask her to talk about what she is doing as she is doing it. Of importance, she should not talk about why she is doing x or y, but instead talk about what she is actually doing or planning or doing next. So, if I am surgeon and I am evaluating a surgical field, I would not say why I am inspecting the field, but I would say that first I am searching for a given nerve and that it is not here and so I am searching there, and so on. The whys will come later, but think aloud is all about what exactly experts are doing at any given point in time. How does that help you create immersive training environments? Well, if you know how an expert or a group of experts act when faced with a given problem within a certain situation, you are beginning to understand what exactly a student should know in order to act as an expert when faced with similar problems.

To know what is truly important handicap the problems presented to specialists: Thinking aloud is a great first step, but the problem is that like in the bike riding example, experts do not necessarily know what they know. One way to help them understand what they know is to handicap some of the problems presented to them and then see how they handle that problem. For example, let's say that I present a clinical case to a surgeon, but then omit the information about body temperature. By observing how the lack of that information will handicap her ability to make a decision, I can then infer now important body temperature might be in the whole decision making process.

That's obviously not the whole story. There is much more on how to map what specialists know, as well as all the factors that are external to specialists but that do influence their decision.

[diveincoaching]

From training to coaching to innovating

Training for new skills is unquestionably important, but once you start getting trained it's natural to want to go from receiving training to being coached. Being coached means that you are no longer just getting information is passed on to you. Instead, you are being personally mentored through the whole process, the specific information being filtered and customized to you at the exact moment you might need it.

When it comes to distance immersive learning, the training-coaching distinction makes a sea of difference. While training is what is being delivered by most distance courses, coaching is still a distant and still not entirely achieved goal.

But then it doesn't stop at the coaching. At some point, being coached towards doing something is no longer enough. Coaching might show you how the nuances of a given skill could be honed, but you want to go further. In other words, you want to create something new, and creating something new is far beyond simply learning more and more about a certain skill. That's when innovation comes into place.

My argument here is then that training, coaching, and innovating are three pieces of the same chain. Sometimes you might jump right into the last, sometimes you might decide to stay within the first forever, but they are connected.

From an educational perspective, the first task is to understand this training-coaching-innovating chain. Training is probably easier than coaching, and coaching towards innovation probably more difficult than simply coaching.

So, what exactly constitutes coaching towards innovation? Is it even possible to coach people toward innovation? Is there a set of heuristics that could be used toward innovation or is the very concept of innovation something that simply doesn't match the stifled idea of a heuristic?

Well, while I don't think anybody would have the guts to propose definitive answers, some would argue that a very important heuristic for innovating is mixing different concepts. This idea is all but new. For example, Steve Jobs once said that

"Creativity is just connecting things. When you ask creative people how they did something, they feel a little guilty because they didn't really do it, they just saw something. It seemed obvious to them after a while"

But how do you go about connecting or, to use one of my favorite words, blending concepts and tools? Do you that alone or do you rely on others? Does the environment matter? Does the way you test your concepts and tools affect how you blend them? Good questions, just not sure whether there are good answers.

[diveincoaching]

Immersive skill training: Fun vs. effectiveness

Whenever one talks about immersive education, the immediate assumption is that of a single picture: Students being absorbed in the learning process up to the point where the information is captured. While the absorption part is a given, I don't think being absorbed in educational settings can happen in a single manner. Up to this point I can see at least two aspects to it:

Fun: It's important for the whole experience to be fun, whether it is just plain funny, or exciting, or scary, or something that will get you hooked. Think for example about the best teachers you have ever had and they were likely fun to interact with.

Situational: Specially when it comes to adults, learning with a specific purpose in mind is important. Therefore, if the learning occurs in a situation that is connected to whatever students foresee as being connected to their ultimate goal, they will likely get immersed. For example, if my goal is to learn Spanish in order to be able to better connect with my patients, a learning environment where I could interact with fictitious patients while learning different aspects of Spanish would definitely get me hooked.

[diveincoaching]

The Ladder Framework for skills training

Was reading a very interesting post earlier today about the constant practice schedule by members of the Phish band. As I mentioned in the post, the concept of constant practice immediately raises a connection with something that was passed to me by Bob Eberlein as the "Ladder Framework" for skills training.

The concept is quite clever and goes more or less like this:

As the Outliers book proposes, expertise is the product of a ton of hours doing a fairly specific thing. At that point you reach a kind of plateau where very subtle nuances in the activity you chose. Some people would argue that the magic number is 10,000 hours -- I would call that generalization bs, but I won't go there, at least for now.

But what this kind of encompassing theory misses, I think, is that it is not only about the number of hours you spend doing something, but it is also about the level of activities you spend your time with.

Here is an example: If a cardiac surgeon spends 1,000 hours conducting surgical procedures, she could do each one (a) from start to end, or (b) do the first half from start to end and then let her assistants do the initial part and then simply come in for the most important portion of the procedure, or (c) could do the first 1/3 herself, come only for the most important part of the procedures for the second 1/3, and then for the last 1/3 she not only comes only for the most important part of the procedure but she becomes a national reference and only operates on a very complicated heart conditions of a certain type.

In my previous example the number of hours and the division in thirds is irrelevant as it will vary from field to field. The important concept, however, is that there is a ladder progression as she goes from (a) to (c). In other words, once she has mastered the easier portions of a procedure, she moves on to focusing exclusively on the most complicated portions of the procedure, and from there on the most complicated portions of the procedure of fairly complex cases.

But why is this called a ladder? The ladder comes from the mechanism the surgeon is using to have her procedures filtered to more and more complex types of cases. In other words, she initially has her assistants taking care of the most basic parts of the procedures, and later on her colleagues referring complicated cases to her

Why does it potentially work better? Primarily because each surgical hour of this specialist is being used with cases that are pushing her skills to the next level. In other words, in an ideal Ladder Framework no case would keep her skills on a steady state, there would alway be an additional challenge that would make her connections among concepts that she hadn't made before, talk with other specialists that she would not have talked otherwise, use tools and devices that she would not have thought of using, perform manual maneuvers that she would not have performed if she were just doing cases with the same level of difficulty over and over again.

From my examples on the types of connections she is making, it is probably obvious that I am coming from a situated cognition perspective to skills training, but that's a whole separate discussion.

[diveincoaching]

Persona in course development: top-down, bottom-up, and n-of-1

The concept of persona is widespread in marketing and software development, but here I will argue that personas are extremely helpful to design courses: Rather than designing a course to a fictitious person with a multitude of general characteristics, you pick one specific person you know and design the course to that specific individual.

Why? Because it is easier to create a course to a real individual rather than this amorphous average person. For example, if a question arises on whether a certain feature would or would not be useful in the course, you simply ask whether that persona would like to have that feature or not. If the answer is yes, do it, otherwise discard it.

Now, although what I just described is the most classical definition of a persona, I will call this simply one out of three possible personas. Specifically, I will call this previous description a top-down persona, or a persona that was developed out of our general beliefs about who the average customer would be like.

In contrast, a bottom-up persona is the persona that emerges from a data set with a large number of people. For example, if a course has three different versions, then the persona preferring the first version of the course will be characterized by a certain age, gender, income, etc. Once those characteristics are known, then a specific person with those characteristics will be chosen to represent the actual persona.

Bottom-up personas can be created through common statistical and data mining algorithms, and can be more detailed than the top-down persona. That said, bottom-up personas can only be defined once we expose a group of people to a course or to a pretty reliable description of what that course will look like and do. In other words, without a course or a proxy a bottom up persona cannot be created.

The third and arguably the most interesting type of persona is the n-of-1 persona. In this scenario an individual is submitted to a series of randomized tests where the persons' response to certain stimuli within the specific course. For example, if I want to know whether a certain person likes videos or texts within a course, I first expose that person to a video and then later expose her to a text. This experiment could be repeated a number of times to ensure that our information on what that person likes is accurate. We can then proceed to test other characteristics. The n-of-1 persona is probably the most reliable as it is the direct result of experiments with a given individual.

Of importance, there is really no ranking among the three types of persona as the three are used in conjunction. You simply keep creating them as you go and then combining as needed while developing the course.

[diveincoaching]

Communicating cognitive schemata to novices

There isn't much of a question that, in comparison with novices, experts have richer cognitive maps. In other words, not only they have a larger number of interconnected conceptual nodes, but the richness of the communication among their conceptual nodes is greater. There are some excellent resources about this field out there, the book by Stephen Downes being one of my most recent discoveries.

What follows is probably described and even demonstrated by others, but here are some thoughts that occurred to me while going over this material:

Experts are likely distinguished from novices not only by the number of conceptual nodes and their connections, but also about how they display the hierarchy among these nodes. So, for example, a certain node clusters are centered around a concept that is considered more important by the expert, and it's this central concept that then connects to other regions of the network.

This central node could well be considered a latent node, since it connects a number of items that are directly measurable

A big bunch of the literature doesn't make use of the extensive visualization resources available through social network analysis software such as R and its amazing task view on graphical models

Although classical elicitation methods such as think aloud and problem handicapping are used in the cognitive task analysis literature - see for example Schraagen's book, it is interesting that these evaluations are not placed within specific situations or contexts. In other words, would be interesting to see the situated cognition literature join the wagon.

Fuzzy cognitive maps should definitely be part of this discussion.

But still, the most important and unanswered question is how, once these cognitive maps are appropriately prepared and represented, how do we use them to train novices?

[diveincoaching]

Lean course development and the need for hedgingCourse preparation, MVP, agile iterations, and the need for hedging

The more I keep thinking about distance course preparation, the more it seems to me that it necessarily has to work as a lean startup. Here is why:

You can't wait for perfection, you have to launch early, with an imperfect and yet functional product - the Minimum Viable Product

You have to keep iterating based on customer/student input

So far so good, but this is all old news, part of the lean philosophy, the only twist being its application to the creation of online courses. But then there is another aspect that is somewhat newer: If you are going to release a course that is rough around the edges, then you have to hedge expectations.

For example, if you release a course that is not at it's very best state, students will think that they are not having their money's worth. How can you hedge then? Well, here are some possibilities:

call it a beta

meet with them prior to them starting the course

be present -- even if it is through a hangout -- when they start the course

get their input after they complete the evaluation

give them a discount or, maybe even better, give them a gift such as a t-shirt or something along those lines

[diveincoaching]

NIH PCORI and the education of the next generation of biomedical researchers

I recently had a chance to participate in a round of reviews for PCORI’s funding proposals. Incredibly interesting process, made me think about a number of how their new rules will mold the new generation of biomedical researchers. Some thoughts about the pros and cons:

The only question I had about the involvement of patients and stakeholders is why haven’t we done this from the very beginning? This was, without a question, one of the most important moves in the history of biomedical research, so glad that this came to fruition.

With a renewed focus on generating true change in healthcare or policy rather than simply generating yet another scientific publication, researchers are now more focused on creating technology than ever before. While well-intended, the university is ill-equipped to do that without partnerships with entrepreneurs. Not only scientific methods are not adequate for bringing technology to market, but they also can hardly lead to sustainable technologies beyond the grant period.

Scientists don’t know how to communicate with patients and stakeholders, and patients and stakeholders have not been trained to communicate with researchers. We need better training to both sides.

With the increase in number of people discussing proposals, the time for discussion is just not enough. We likely need to make use of Web applications to deepen the level of discussion prior to the in-person meetings.

In sum, a number of challenges remain, but the as I said in the beginning the inclusion of patients and stakeholders is absolutely outstanding, definitely a step in the right direction. Looking forward to seeing how this will evolve.