Kent Bye

Extracting and redistributing knowledge

[kentbye-blog]

Using the Core Protocols and Commitments with Agile Teams

Jim & Michele McCarthy created a Set of protocols and tools for creating high-performance teams that deliver on time. http://www.mccarthyshow.com/online/

A site that someone points people to is Adam Foyer's Live in Greatness - Idea of becoming great. Find how to get there. http://liveingreatness.com/

Core Protocols: http://liveingreatness.com/files/core-protocols-3.03.html

The core protocols is difficult to spread because it is scary. People don't like to feel vulnerable. There's a bias towards action.

There's a cycle of: Generate Goals > Create Ideas > Take Decisions > Take Actions

Shared vision. Have an ecology of ideas. Taking decisions. Translate those into action.

11 Commitments basic rules that people agree by. Demands a high level of intention and it doesn't guarantee success.

1. I commit to engage when present.

2. I will seek to perceive more than I seek to be perceived.

3. I will use teams, especially when undertaking difficult tasks.

4. I will speak always and only when I believe it will improve the general results/effort ratio.

5. I will offer and accept only rational, results-oriented behavior and communication.

6. I will disengage from less productive situations

7. I will do now what must be done eventually and can effectively be done now.

8. I will seek to move forward toward a particular goal, by biasing my behavior toward action.

9. I will use the Core Protocols (or better) when applicable.

10. I will neither harm—nor tolerate the harming of—anyone for his or her fidelity to these commitments.

11. I will never do anything dumb on purpose.

http://liveingreatness.com/files/core-protocols-3.03.html

There's a Core Protocol Boot Camp

Protocols to deal with how to deal people who are living up to the shared commitments

Number 1 protocol is the check-in. Came from Men's New Warrior weekend from a marine Jungian analyst.

Check in is where you list how you're: Mad, Glad, Sad and Afraid. Rules around sharing. Don't interrupt. Implement a cone of silence while people are sharing. At the end, then you say, "I'm in." And then people say, "Welcome"

The commitments are short and pithy, but there's a books-worth of protocols.

Some Demonstrations of a Check-in:

"I feel Glad and inspired that this is happening. Core protocols codify a lifetime of wisdom and that they work. I feel glad and joyful being at the conference, and meet and learn from amazing people. I'm Afraid/Glad (excited)

"I'm glad b/c conference is fun. Glad didn't think it wasn't going to happen. I'm afraid because I'm thinking of two agile open NW events to happen in Seattle and the associated work. I'm sad. I'm in."

"I'm glad this is happening. I'm mad that Jim isn't here. I'm sad that if people don't pick up and share it, then it may die over time. I'm afraid that storms prevent going home."

If you feel uncomfortable or unsafe, then you're free to say pass. You should also say "I'm in" even if you don't check in. You can also check out. There's also a group check-in.  

If you're passing on a regular basis, then you may not really be in and present if you're not.

Check out means that it's a self-care oriented protocol, and you're going to physically leave the area. There's also a commitment of "I will not offer and tolerate inappropriate emotional behavior." Don't have to have a reason to check-out. Commitment is to be supported. There's a bias towards action. If you insert drama on purpose, then it slows down the team. This gives everyone a tool to express you emotions without adding stories.

Checkout isn't just about anger. If your time would be spent better elsewhere, and you feel held hostage, then there's the Law of Two Feet type of checkout that is there as well.

One time got fired for checking out. Boss said that is was unprofessional for checking out. These are easy to adopt, but if they don't look at the core commitments, then you're not really agreeing. You can't have one without the other. Everyone has to agree to the commitments. It creates safety when people do.

Specifically valuable to check-in when you're angry.

Started to learn protocols and went out into Web of Commitment. Your whole team needs to use it, and you can't just have a few do it. Create personal interdependencies between people otherwise some of the protocols can be taken the wrong way. Exchange personal information and help people to self-reflect and build trust, and create a situation where they can start to use the core protocols.  If they're not in the web of commitment, then they may take it wrong. These commitments are a culture hack for interacting with each other. If you go back to a different team, and if they're not "Booted" then it'll be difficult to work with them.

If you can communicate to your team how the commitments work. In his work, they use the Core Protocols of: Pass/Unpass, Decider and Resolver. Half were booted, and you can use those as well as they can agree to those commitments.

The real power of the core protocols is in the web of commitment, and be willing to make the jump to be vulnerable unless they're innovators and early adopters. More and more in the agile community are starting to 

Web of commitment is built upon the personal alignment to get what you want by asking what can get me past the barrier to get to a universal value. This is a perennial philosophy where people that agree that love and courage is a good thing. Personal commitment to trust and to ask for help from your team to get love and get courage and wisdom and integrity. It seems simple, but you're practicing that. The whole premise is that you can source them at any time, and point at them as a source of problem. Likely that if you bring love and courage to the team, then it'll likely resolve the problem. It's extremely powerful.

It's very difficult to say without a bootcamp, and it's difficult to get a team to commit to doing it. It's difficult to boot. But's it's easier to introduce the principles and get people interested in them.  Helps to get in touch with emotions and create personal alignment.

Decider and Resolution are a great protocols that are biased towards making a decision.

Have use the protocols in actual chartering. Charting session to come up with the core commitment: "Test knowledge, dev and knowledge management skill are of equal value"

The perfection game is an effect way to take an idea or proposal to get feedback and improvement. Gears things towards what is actionable. Where is value? What actions can you take to improve that you already have? Perfection game is very positive psychology by focusing on where you can add value. If you add this it'll be better. Not giving negative feedback.  You can apply it anywhere.

Rate what you did on a scale from 1-10. What it would take for it to be a 10. It means that you don't have any ideas what you can do it'll be better. How much value and improvement can you add to whatever it is that you're perfecting. If it's a 5, then what we double it to a 10.

Ask for help: Will you help me improve this diagram? It's best for the performer to ask for help and not recommend.  One a scale out of 1 out of 10, his feedback may be 8 out of 10. And relative to that, then how every much that you add to that diagram.  Give feedback.

It flips the normal rating on it's head. If someone has rated something a 3/10, then that's really exciting because that means that they have a lot of ideas for how to make it better. If you get a low rating, then you may need to think more about how to incorporate a lot of that feedback.

Unpass - Changed my mind after that you've passed. If you check out, and have to leave, then you have to check back in

Any ask for help is an honest ask, and a legitimate response for no.

Asking for help. It's simple, but important: "Will you" Try to avoid being rescued.

"Will you be willing to..." is a part of NVC. Otherwise it's a demand. If people have been booted, then you'll use "Will you?"

Lots of NVC analogs to Core Protocols. NVC has: Observe, Feelings, Needs, Request. Core Protocols has: Investigate, Check-in, Values, and Asking for Help.

NVC Observe is analogous to investigate. To generate ideas. Investigate is a commitment to curiosity. Don't use socratic method, and it's about being open to understand. Feelings is like the checking part. The values/needs is analogous is about the personal help. Ask for help is the request of help at the end.

Protocol Check. Can someone help me to check to see if Harold is following the protocol? You can pull discussions to a stop so that you can check the process to make sure that.

Cost for a Boot camp can be up to $5000, but the McCarthys are incredibly generous. There can be a barrier with the cost.

Doing a 1-on-1 checkin has been incredibly helpful.

Dev team does a quick check-in.

Do check-in at the beginning of retrospectives and kickoffs, and sometimes when the energy gets low. Check-ins tend to go really quickly with shared context and when you're doing it really frequently. The more frequently that you do it, then the deeper that you can go and the team gets closer. If someone is really pissed about external factors, then you understand a lot better if people snap because of what's happening in their lives.

There was a team that adopted 4-5 protocols, but decided against the check-ins. Then had a retrospective where someone broke down and cried because they didn't feel heard. Then the team added the check-ins.

One failure mode that see about structured ways of dealing with interaction and emotional territory is that it can become a stick to beat people with. Ignore the essence of what they're saying because it doesn't follow the protocol, and it becomes a shield. Seen it happened with other systems. 

Where can Core Protocols go wrong? If people don't commit to the commitments. It's a wonderful framework, but it's not the goal to be compliant with the Core Protocols.  When you're really moving you don't need to use it.

Doing a lot of NVC and non-violent communication in an intentional community and there was a lot of doing NVC AT people.  Put off by jargon like "incoherent emotional transmissions" and "booted." People love to name stuff and build mutually supportive practices, and then there's an exponential growth of jargon and terms that is alienating. 

If someone's being a jackass, then check-out and leave.

Read the Core Protocols book, and the language and terminology can be a put off. Lots exciting stuff, and it's like software for your head.

Seems young and new enough that it's not practiced in a lot of places, and it's implemented in reasonable ways. But if it gets huge, then people can be really pedantic and crazy about it.

One of the commitments is "Use the Core Protocols OR BETTER."

When it didn't go well. It has been used an open space conference before pass, and checkout, and before the law of two feet. "Meal Decider" is a hack of decider, which is a biased to action. Normal too much. Propose: Place and location and the time. If you're not in, then you have to propose an alternative place, location and time. It's not important to capture people's concerns with Meal Decider. It's closer to consent-decision making rather than consensus. Don't do decider with more than 5 people is a part of the commitment.

[kentbye-blog]

Fast Thinking vs. Slow Thinking

YouTube Video that gives an overview of Fast and Slow thinking:

"Brain Tricks - This Is How Your Brain Works"

http://www.youtube.com/watch?v=JiTz2i4VHFw

Thinking, Fast and Slow by Daniel Kahneman

http://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374275637

FAST THINKING

* Doesn't tire me out

* Associative

* Intuitive

* Narrow

* Respond more slowly to losses than gains with fast thinking.

* Ignores risk

* Often gets to wrong answer

We're wired to be reactive and associative. It's our natural tendency. We avoid slow thinking because it's hard. Easy to flip into fast thinking, and we mimic others and react without thinking.

Slow thinking is thinking it through. Grinding it out. Slip into fast thinking because it's easier. We don't have to think deeply and react to the charging tiger and it used to be appropriate, but now it's not so appropriate. 

The practices of Agile and TDD causes you to slow down. Reflections and retrospectives are another way to slow down as a group.

The question for this session are the dangers of fast thinking. Fast Thinking narrows your focus.

Acronym of "What You See Is All There Is" - WYSIATI

If you're not looking for evidence outside of your normal scope, then that can be dangerous. Our confidence is based upon the quality of the mental story that we can construct. If there's missing evidence, then we don't even go looking for it.

Fast thinking leads to bad results, but we can't always control it.

What are some practices for slowing down and move into a more logical place?

POMODORO TECHNIQUE

Pomodoro technique can help to slow down. Helps to prevent burnout. 25-minute increments, and then take a 5-minute break to walk around and reset your thinking about an issue. Helps to build in breaks and re-think about an issue. Jot down 5-6 things down before Pomodoro. Step back and think through problem more clearly.

http://www.pomodorotechnique.com/

Overview PDF: http://www.pomodorotechnique.com/download/pdf/ThePomodoroTechnique_v1-3.pdf

Mindfulness is a process cultivating attention. Taking a deep breath and pausing.

Overconfidence in estimations are fascinating. It's bad for a group is when the group feels at ease with the solution, and so people may jump to the wrong conclusion.

Scrum framework is scalar. Slow thinking happens it at the beginning at the sprint, and then team hits the sprint and moves into fast thinking. Take a breath for sprint review and the retrospective. Slow think then Fast think to really slow thinking and restart.

If the retrospective is stressful or exhausting, then there may be something wrong. Encourage slow thinking and ask the deeper questions.

Post a "How when you know when you're done?" on a post-it note on your computer as a reminder. It pulls a developer out of a thread. Step back and reason what needs to happen in order to get the task done.

Start to notice when you're triggered in unsafe situations like "In the line of fire" in order to build up habits. Take that outside the workplace and notice places where you might need to take a step back.

Can you improve your ability of Fast and Slow thinking? Yes, you can work at it until you it becomes habit. System 2 informs system 1, and associates correctly. Times that you can't get it wrong. Fast thinking is also happening all the time. Meditation can help raise awareness.

Expertise: Repetitive task, then you can improve your intuition. Build skill if you doing similar things over and over again. 

Does scrum support slow AND fast thinking? One person disagrees. People identify patterns and process, and people can start following that process without thinking.

People are very bad at understanding risk and how we respond to risk. If you slow down and think about the true risk, then you'd be paralyzed.  Leadership with pervasive optimism and risk aversion and avoid talking about risk is a cultural thing that can be a bad thing.

Ask self to slow down.

Thinking Fast and Slow. Priming is a powerful phenomena.

Cognitive Behavioral Therapy suggested to keep an Anger Journal: Notice irritated and annoyed, and then recognize physical signs in yourself. Once you recognize, it then be able to think slowly.

What if management wants to make decisions around data instead of from the gut of the managers? Believing what it takes to do something on the outside, sometimes we don't look to the outside.

Even in the face of overwhelming evidence for logic, you have a lot of power if you prime people correctly.

Like when you get triggered in PTSD, and then need to notice it and then have a plan. Do the slow thinking in advance so that when the fast thinking happens, then you know what to do after you're triggered.

Meditation for anger management in prisons. Your not weakening slow thinking, allows the slow thinking mind slow down and bring fast thinking to the conscious level. That's very valuable for us.

Woman had a son and took meditation at the University of Oregon. Emptying mind to be in the moment. It's challenging. Goal isn't always to just to empty the mind.

Talk about insights from the Blink book about the studies about decision making. Make a decision quickly within a quarter of a second as to whether an object being held is a gun or a phone. If you have less time, then the person's race will be the determining factor with the fast thinking. So much depends on the associative context. Depends upon how much time that you have to make a decision.

Took a "Implicit Bias Test" online about Asian bias at Harvard. European vs. Asian face and then measure how fast and how many errors. Then switch to objects. Easier to group things that are together in your mind.

Police practice walking out of the car slowly when approaching a car to get into the mindset of slow thinking. WHen you're in fast thinking, then you're slow thinking is not in effect. Walk much slower in Network Operations Center. [laugh]

Partnered cops will end up much more in violence. Does that hold true for Pair Programming? [laugh]. Switch out pairs and not use the same pair all the time.

Give pauses so that people who are more introverted can participate in it.

Comedians use the fast and slow thinking. Alternate between the fast and slow thinking.

Things that make something a joke. When you know the context, and the punchline throws you outside of your associations.

Being drunk helps to get out of fast thinking.

In the Constitution, the founders built a system that was slow and deliberative because the fast thinking causes bad snap decisions. Think through checks and balances through unit tests.

[kentbye-blog]

Applying the 5 rules of Accelerated Learning

This is an continuation of the 5 rules of Accelerated Learning session from yesterday. The notes to that session is here: http://aonw2013.posterous.com/5-rules-of-accelerated-learning-by-willem-ses

Simple rules create behaviors that drive complex systems.

For example, for Flocking is 3 rules: 1.) Steer towards center 2.) Avoid flockmates 3.) Maintain speed and direction with your flock.

Rules are not controlling complex systems, but to help providing driving inputs. You can tweak the knobs and pull the levels. Are you going to amp or dampen each of these 5 inputs.

1.) Alive - Serve Aliveness to Generate Energy for Work. Not an option.

2.) Fluency - Pursue fluency to expand capacity. It's an Agile Value to know more today 3.) Signal - Amp signal to initiate action. "Let's go!" If participants don't know what the vision is.

4.) Focus - Narrow focus to drive progress to increase proficiency, and design the environment accordingly.

5.) Environment - Design the environment accordingly.

Asking participants to raise your hand is a way to amp the signal. Notice process. Like if there's laughter, then that's a good sign. Smiles is another feedback. What wo

What has worked with the team room? 

* Separation from other teams - Increases focus. If teams are not separated, then there's much more distraction and incidental things happening. Converse point is that you can get more cross-pollination when two teams are close to each other. Sweet spot between focus and cross-pollination.

* In a team room, if two tables in a room, then have a comfortable buffer and space between the two tables. 

* There's always constraints. You can design around it.

* Tell team to put desks where they want it. Started right next to each other, too distracting. Then really far apart, but then no conversation. Can see how the team adjusts their own environment. Some have movable walls. 

* Open space doesn't work as much if there are a lot of different teams. Couldn't have conversation because of sound issues and it was too distracting. Need privacy for difficult and frank conversations. Open spaces can get people to get sick more, and they tend to make friends less. It affects Focus: Ability to make friendships is harder. Social investment is diffused.

* Software work is knowledge work. Knowledge work is learning work.

How to make your space better in the context for working with a virtual team.

There's an identity element to teams, and having a virtual team. There's less of a emotional connection or a symbolic flag that it's a team effort.

* How much aliveness? 5 on a scale of 1-10

* How much fluency - in context graceful action and competent action is happening? 6

* Signal - Paralysis vs. Kung fu chop to the brain? 4

* Focus - 4

* Environment - How supporting that supports all other? 8 (co-located, bull pen, places to go, good cross-pollination) 2 in the other location -- Averages out to 3 because distributed company.

If you could change of the factors, which one would it be?

How could you  bump up the aliveness?

* Bring the remote person into Portland in 3 weeks. Like an offsite, team meeting.

* Have people rotate into other groups and create isolation to build empathy for the remote workers.

* Individuals are geographically in two different places. Group chat rooms. Use Skype for standups. Bad bandwidth in South Africa. Temporally work the same hours.

* Put as much conversation in Campfire and share photos.

If you created isolation, then what are you looking for? What would happen at the end of the week? Help to think of other experiments. Empathy is a great tool.

Best learning is diving in into a real situation rather than doing a simulation.

Aliveness is your enegry gage, Facial expressions, energy in a conversation, Hoping that you'd feel more energy and be more engaging. 

Lots of queing and signaling that happens in a co-located environment. Express feelings as soon as possible to avoid resentment. Express when you're upset and not including. So increase the signaling between team members. See where signaling is broken. If you don't know when you've lost it, then

Willem would call that "Fluency" rather than signal. It's already being generated in the space, but being done awkwardly. 

SIGNAL: When working remotely. Didn't physically occupy any space, and created an abstract thought. Build a physical model with a large monitor. Turn on the camera at all times. Hi-definition camera using Skype and it's on all the time. Side shot to mimic sitting at the desk. Super powerful. Help to make the abstract real. 15-inch monitor didn't matter. Try putting a roomba and drive it around. But had to go the full 42" monitor and spent his own money to do that. He did that without asking. Bought an array microphone and velcroed on the type. Tool on the other side to automatically answer the Skype call. Can listen in to the water cooler conversations.

Roomba battery only lasted for four hours. Got stuck in the hallway. Shrunked head TV think was an overreaction and used to be projected Wizard of Oz style. Too much Signal was the project. Need to find the sweet spot. Then measured himself to make it more life-sized. Now it feels like to them like he's actually sitting there. Getting to more subtle elements of ALIVENESS. People wouldn't engaged with the avatar. The bosses just wanted to use a conference call. Need to see mouth when speaking, and when speaking, not sound like modulated from the conference call. Couldn't get the bosses to turn on their cameras to turn on and use them. 

If there's more frustration, then theres more noise and greater signal drop. These are all productivity design points. 

Avatar isn't religious. It's humanity and an intervention that works. The bow on a present, if a person gives a child that's a knife. Not a great chance that they'll lose it. But if you give it to the aunt or uncle, bounce it off a peer, and give it to their colleague and increase the aliveness. There's a deadzone of signal in status gaps. Find the route where the energy is, and that's aliveness. 

Critique how and why it works according to these tools.

Telepresence centers are being put in all over. Anyone lower than a director-level could not get into those rooms. Had Seattle have fixed space in his own situation. Orient it out the door, so that he can catch people as he walks by.

When you're tweaking knobs, then there's a time difference for when you can see when it takes effect. Also you should notice some positive change right away, even if it's within yourself. Want a rhythm and see the results. Do an experiment and see what happens. May not know the timeframe.

Hawthorn effect of the Bosses are Watching. Turn down lights. More productive. Turn up lights. More Proctive. Shift anything was that the bosses are changing things, and people notice the CHANGE rather than what was changed.

How engaged at lunch? When people get home? How to nudge up the needle? No just HOW MANY X THINGS GOING OUT THE DOOR.

Without a design habit, we're all design thinkers. Lots of people don't think about how to hack something. Theres's only right and wrong way and "I'm in charge." For anyone who isn't a design thinkers, then tell them that aliveness is the best gauge and there's research to gauge how much work can get done. It's a systems way of looking at things.

Use aliveness as your gauge.

Every moment is an excuse to quit.

Use fluency to point at useful tools to boost it. Use tools to boost aliveness.

What do people do to increase aliveness? Interactive. Laughter. Number of questions people are asking. Engaged and curious mind. Dynamism. Spontaneous collaboration. Fluidity and synergy and emotional connection with attention going to same place at the same time.

Can also be in a place where you can cut through the tension with a knife. Can feel when the environment is not alive.

Creating positive noise and activity: Snack table and potluck materials. Beer and alcohol in context and when appropriate.  Creating a variety of different options. Agile clinic in one clinic. Job board in another place. Open space.

Law of two feet creates an opportunity for fluency. Three ring circus creates a chaos, and 

Have a topic come up a lot over time? Then that thread has a super-learning effect. That's called "Interleaving" and variety: When learn something new, then you come at it at a bunch of different directions.

Looking at the Design at Open Space

The Law of Two Feet is a Skill. People don't want to interrupt, and were conditioned. Be rude. Force people to leave. Don't leave at other time. Have 15 seconds between leaving.

People also don't speak up when there's a tension in the room. If you screw up, then give it to me straight. If screw up, then it's hard to really tell people when they don't like it.

Learn to cut your losses, and leave when feel compelled. Sometimes too many people for the conversation you're trying to have.

A good design does not rely on Virtues of everyone. People are results of natural selection.  There's nothing broken of any of us. Design an environment for humans.  Environments are filled with tensions that need to be resolved. Resolve them towards your goal and what you're trying to do.

[kentbye-blog]

Product and Project Visioning with the Product Vision Board

The Product Vision Board is the highest-level document before the execution begins where it's broken into the stories through grooming stories. This could be set up as a 1-hour product overview meeting with the team. It can be expensive, but it's an exercise to help create alignment. It's a kickoff exercise for the product to get everyone in the mindset that we're working on the same thing, and create the vision and create the buy-in.

Can alternatively start with the mission, and think about the team is going to need to use. There needs to be alignment. 

Product owner will come up with a first pass of this Product Vision board. Then take it to the stakeholders. Then the product owner will take it to the development team to flesh it out even more. Constraints are added and fleshed out, and constraints often come up in implementation.

The idea for the Product Vision Board from Roman Pichler, who is a product own blogger at http://www.romanpichler.com/blog/ More info on "Working with the Product Vision Board" is here:

http://www.romanpichler.com/blog/agile-product-innovation/working-with-the-agile-product-vision-board/

Pichler blogs from the product perspective with pragmatic information on analytics and business model on a consistent basis.

This is to help create alignment on your teams. Easy to work in a box, and go work on it without caring about what else is going. This will help you transition from Conceptualization to Product Team and implementing with Scrum.

Product Vision Board has four vertical columns (Users, Needs, Features, Values) with a header portion at the top for Mission Statement. Constraints are listed at the bottom of each column. Specific, actionable and time-based goals. Say what they're going to do and when they'll do it, and like a Mission Statement for what going to deliver and why.

Mission Statement: Pizza shop that wants to create a Windows app.

First Column is the User. You can either create a user persona, which is identified with a specific person. But you can get too specific with user persona, and he uses "User Categories" instead. Like a call-center representative. Then list the constraints for that user in that column, like a call center rep won't have high-performance machines.

Second Column lists the needs of the users in order for them to be happy and solve their issues. If the user's computer is constrained, then the developed application needs to be lightweight. Let the team write on post-it notes, and then let the team put it on the wall. Gets users to get more buy-in, ownership and autonomy.

Third Column is the Top Five features. Feature constraints helps to identify things you haven't thought about.

Fourth Column is Values.

Let's do an example. Background: We're a bleeding-edge pizza business. In college town with lots of smart phone users. Build an app where you order and deliver a pizza without talking to anyone.

Users

* [Add all of the users in the value stream]

* College students

* Pizza makers

* Delivery employees

* Business owner

User Constraints

* Need to own a smart phone

Needs

* Pay for my pizza

* Get my pizza

* Chose pizza

* Order in bulk

* Metrics - Think about how you're going to measure what you're doing so that you can get alignment. Book: "Liftoff: Launching Agile teams" talks more about that.

Constraints

* PCI or other compliance

* Within store hours

Features Top 5

* Easiest to define the Features

* Order Pizza

* Enter address

* Enter phone number

* Display pizza queue to the

Values

* [Hardest to define]

* For example, "Simple app" is too vague. Needs to be more measurable and definable.

How to create the "Why?" criteria

* Create alignment with the team. Sometimes a presentation, but like for it to be a collaborative exercise.

* Provide Reference - Post it somewhere where it is highly-visible so that people know what they're working on.

* Opportunity for thinking through your project - Helps to create the ability to think through what you're delivering.

* Crossing the Boundary from Planning to Execution Phase -- Getting ideas to specificity. "Make it faster" or "Build a simple application" is not specific-enough

[kentbye-blog]

Helping Product Owners Write Better Stories

Here are my notes from the Agile Open Northwest session on "Helping Product Owners Write Better Stories"

Stories that are written are a contract between business people, developers and QA. What are we trying to do? What are the requirements? Puts everyone on the same page. Everyone will know what will be delivered at the end. Helps to put everyone on the same team.

It's not Us vs. Them. Product owner is part of the team. They can't write stories without you. They need to be vertically sliced in way that make sense to the developers. Story can't come into the sprint until it has 3-4 acceptance criteria. Need to start grooming the stories before the sprint planning meeting. Team should have seen them as epics beforehand to break them out into vertical slices and it makes sense, and can be proven that something is better than something else. It's unique for each team.

Pre-grooming on really big stories. Epics are getting more complex. Added a new pre-planning and backlog grooming meeting. Meet up with the product owner in the middle of a two-week sprint. Helps to bring up technical requirements and things that you need.

Can't get to sprint planning meeting without knowing what they're going to be doing.

Groom the tasks that you know less about. Eliminate vagueness.

What level of detail at backlog grooming meeting? Not at the task-level, but at the story-level. Put points on the story and stop. Do all of the business stories at one point. Do 1, 2, 4 (1 week) and 8 (2 weeks)?

As product owner, if he goes too small, then it wastes time. Seems contradictory to look at stories at too small of an increment.

Getting the product owners to start to ask the right questions. Acceptance criteria can start to flesh that out. Need to push back if you don't have good acceptance criteria to get at the root at the problem that you're trying to solve.

Velocity has increased, and have done more within scope. If the product owner has too ambiguous and vague, then it can slow down the team.

At pre-grooming, they try to get the stories to small increment to be broken down later.

Product owners could find an IT buddy to pair with sometimes to get ramped up on the technology, and vice versa. 

What has worked for other people? Do headlines only. The PO and designer will create a headline. Define dones, and then it moves to in progress. Devs want to be a part of the problem solving process. Example of a headline: User wants to add their +1 to an event. They've signed up to an event, and they want to add a guest.

Product owners will put in their acceptance criteria by listing out the requirements for completion. Sometimes it's vague and not specific to engineering. Work with jobs between hospitals. A Job must be in the system within 5 minutes. If you have 10k jobs, then they have to be done within five minutes. That is helpful requirements for the development teams.

Not helpful to have a minimal descriptive acceptance criteria in Jira.

If there's a challenge with time, then there is a sense that it'd be helpful to slow down.

Would like developers to be able to talk the talk of the business, and the product owners start to be able to be more sensitive to the language of the development team.

Needed to teach the product owner that you're on the team. You're not higher than anyone, but that you have a signoff of the final product.

What happens if product owner can start to write solutioning stories? It's the team's decision to decide how to do it. It limits the scope of thinking if they're thinking too much about the implementation. 

Trying to eliminate carry over from sprint to sprint.

Product owners determine the priorities.

Sometimes a product owner can think that they know the full scope of technology, and start to underestimate the amount of time required to fix it. "It's just a SQL update, that's just a 1."

Product owner is an ex-developer who is tempted to start in and start coding, and have to bring him back. Don't tell the developers how to code. You developed and know the code base, but it can pigeonhole them into a corner. Not helpful to have stories that are too technical. Not able to meet requirements of all of the customers.

The "SO THAT" portion of the story is the most important part of the story because it encapsulates the business value. Ask the question: What problem are we trying to solve here? Sometimes it's helpful to start with the "So That" portion.

Break down the features, look at the acceptance criteria statements and see what tasks can be grouped into stories. Once you hit all of the higher-level requirements, then you can deliver it.

Horizontal vs. Vertical division to deliver a Minimal Viable Product? Sometimes have engineering department write it, and then hand it off to the Product Owners. Sometimes needs certain steps first like researching new technology and build a preliminary prototype with the team. Engineering can write their own stories for those. The Engineering Department can start to own stories that are given to the IT department.

Can be considered technical debt to keep up on maintenance, and then negotiate a budget for time to work down the technical debt.

Did an innovation sprint over the holidays, and some of those features ended up in the end product.

Negotiate can't deliver this until certain infrastructure is developed. The original feature may be a 1-point feature, then it may be delayed by a few sprints if there are things to be done first. 

Tie infrastructure and tech debt list associated to specific features. Can also bake infrastructure into certain features. It's like a tax where it's more expensive at the first, but then additional features are cheaper.

May have an absentee product owners who have a designated proxy. Product owner doesn't want to participate in the product development process and get another engineer, and get stories that are more solution-based rather than business-value based. Also the proxy often doesn't serve as a clear communications channel to and from the product owner.

Product owner can also get blamed later for not pulling on the reign harder. Needs feedback from developers as well if that's the case. Developers don't want to commit based upon vagueness. Setting a date and a minimal viable product, then they usually hit those dates. Without a date, then things really start to creep.

Having a proxy product owner is an organizational impediment that the scrummaster to help alleviate.

Sometimes a product owner would cause the developer team to go slower because the requirements were so off. Then she wasn't doing her job, but it was better for the development team.

[kentbye-blog]

Principles of Lean and How it Compares to Scrum

* What is Lean?

Came from Japanese Manufacturing, the Toyota Production System. Edwards Deming was involved.

What is it about the Japanese culture that helped create this collaboration?

Japan needed a lot of help, and they were resourced constrained. Ready for dramatic changes, and Deming was there to influence. Tom & Mary Poppendieck helped to bring it over to 3M, and started spreading it. David Anderson wrote about Personal Kanban.

PRINCIPLES OF LEAN

* Value people. 

* Small batches / Limit WIP. 

* Focus on throughput. 

Achieve flow. Produce inventory. Flow. Focus of value. Eliminating waste and eliminate waste. 

Optimize the Whole instead of local optimization

* Build quality in.

* Mistake proofing

* Root-cause analysis, and then eliminate the source

* Kaizen: Continuous improvement

* Alignment with business ecology. Look at how long it takes to deliver something. Coordinate with partners. US mentality is more competitive and independent, and there's more focus on mutual benefit in Japan.

* Visual management and feedback.

* Theory of constraints - can only go as fast as the bottleneck. Drum/Buffer/Rope. Drum is the constraint sets pace. Buffer helps to maintain consistent pace, and don't want a gap. Ropes (Pulling out finished work).

* Discover bottlenecks through a pull mechanism instead of pushing

* Don't run productions at maximum speed

* Unevenness. Burden. Waste.  Unevenness is like 3 months of dev, and now throw off to QA. Unpredictable. Last item in the queue being too big. Too large WIP.

* 100% utilization leads to excess inventory and waste.

* Feedback loop cycles.

* Single-piece flow is the desired ultimate state. From source, production to delivery with no waste and it never being put down. 

* Talked about Nordstrom innovation lab can experiment with developing iPhone app on the store grounds.

* New New Product Development Game: http://hbr.org/1986/01/the-new-new-product-development-game/ar/1

HOW IS LEAN SAME OR DIFFERENT FROM AGILE/SCRUM?

* Both have small batches. Use WIP to limit batched work for parallel work. Scrum uses time-boxed sprints

* Kanban manages flow expectations in terms of cycle time. Scrum has more of a focus on time and sprints and measures throughput by velocity.

* Kanban has Takt time where you try to match the pace of production with customer demand and the net available work time available. You want the same input and output with evenness. You tend to have more of an infinite backlog with Scrum, but then you need to groom the backlog.

* Scrum sprint tries to minimize interruptions. Lean tries to eliminate expediting fast-tracking because it is disruptive.

* Both try not to overburden the team

* The purpose of flow is to find limits to flow. When you lower the water, you'll find the rocks. Then you remove the rocks and eliminate the waste in a just-in-time matter.

* Retrospectives, Demo and Sprint Planning.

* Scrum transition is more disruptive than a Kanban one. 

* Scrum has Plan. Do. Check. Act.

A3 is an effective problem-solving process 

A3 is a 11"x17" size of paper. Write everything on that paper. If it doesn't fit, then fold it in half. It's like going from a 2-week sprint to a 1-week sprint if you're not getting anything done.

A3 process goes through these different questions

* Problem

* Current State

* Future State

* Root Cause Analysis / What?

* Countermeasures

* Follow Up

More details at: http://www.coe.montana.edu/ie/faculty/sobek/a3/steps.htm

Value-stream mapping to optimize the percentage of time that you're adding value.

Identify customer, and the value is defined by the customer, and anything they're willing to pay for is value, which is one definition.

Recently toured Kaas Taylor software company in Seattle, each employee is responsible for 1 Kaizen per month in the A3 format. 170 employees, and each implemented Kaizen averages around $270 of savings.

Other open questions:

* What can we learn from Toyota?

* Where will it go?

* Lean curious

* Learn about A3

* How to incorporate lean into Scrum?

* How to apply in my consulting practice?

* How to "Lean out" teams and standardize

* When/if to break up dev+test columns

* Lean vs. Scrum. When to apply lean concepts with scrum

* What concepts don't apply to lean? We're doing design

* How has Lean changed your thinking?

Books

Thinking for a Change

Lean Software Dvelopment

Velocity

Out of the Crisis

The Goal

[kentbye-blog]

Myths of Kanban

Notes about the "Myths of Kanban" session from Agile Open Northwest

"MYTH: Kanban teams don't need to estimate"

* Kanban doesn't dictate that you need to do estimates. It's up to the team.

* Most valuable part of the estimates is the meeting, and the conversations that happen during the sizing conversations with the team. As humans and software developers, we can do Small, Medium, or Large. We're not as good as much differentiating within each of those.

* Team morale goes down if there is no progress, and then need to use WIP limits. Big tasks will just sit there and the cycle time goes to 3 months if you don't complete work.

* Used to use fibonacci sizing in scrum, but spent a lot of time without a lot of effort. Found that their 90% of the stories were either 1 or 2 weeks. It's so they sized tasks as big (2 weeks) or small (1 week), and it seemed to work really well. Cycle team is from when it was taken from the working queue to being accepted by the P.O.

* Suggestion is that if you get every task down to one, then don't need to estimate.

* Everything is a story without numbers, but need to break it down to get it down to smaller pieces.

* Write stories the way you want to written, and then track your tasks over time. Doesn't matter if it's big or small as long as it consistent. Track statistics and make sure your standard deviation doesn't fluctuate widely. 

* Estimating can be seen as waste. Do the estimate, but not do anything with it. If you need to improve the estimates, and you get paid according to it, then there is business planning implications.

* Release planning is why you should be interested in estimating and velocity.

"MYTH: You Can't Do Release Planning with Kanban"

* Kanban says nothing about release planning. There is no official way for how to deal with it. However, there is a Monte Carlo simulator at FocusedObjective.com that will take scrum and kanban projects and will return simulations. Sensitivity analysis to isolate defect rate, rework rate and cycle time. They are building this within Lean Kit. Input has to be XML files. Instead of doing a telephone game where at each increasing level estimates have a fudge factor applied. WIP limits can represent staffing levels. Give feedback to which inputs that you can tweak (cycle time, defect rate, WIP limits) in order to get better idea as to when you'll reach a certain point of work completed.

* Release planning could also be seen as useless. But there is also business value to it.

* Team almost never does what's in the release plan if you're doing release planning. Essence of agile is to do what is the highest value and goals that you're pushing for. Decide over time. But need to have a sense of the bucket and value you're trying to provide. 

* Release forecasting is inaccurate for a long time, but even with Kanban flow will still want to have a sense of the goal. Then how to recalibrate with Kanban? Continuous release. Being paid to solve a problem.  Difference between a roadmap and release plan.

* Agile transitions, marketing cannot market things that are not going to be in the feature. Yet, you can also define the task that it isn't done until marketing is done.

* Story acceptance requires that there are statistics done, and so there is an early push of code in order to get stats from production.

* Single-piece flow, it's strange to have a column called "Done done." -- But not if that column is well-defined.

* There are columns that are wormholes like something within the code migration department. You can measure and say that it takes 6 days to get to code migration, but then it takes 8 days within a certain department.

"MYTH: We're disciplined enough to not need Work in Progress (WIP) limits"

* Beauty of visualization is that it'll show you your problems, but it won't fix your problems. 

* There may be a cultural thing to not let people know what is happening, and so they will avoid exposing what it is really happening.

* Value stream and status board only having 3 columns. Will show when things are really stuck. Can start to discover the bottlenecks. Then can start to break out tasks and create more columns. Can have five tea

* Can see average task assigned per user. It used to be 15, but got it down to 6.

"MYTH: You can't do retrospectives with Kanban"

* Kanban is added to existing process, and so if you're already doing retrospectives, then add it in. Kanban will help you to inspect and adapt. Change WIP limits, and change process. Kanban is a good way to get to the how instead of talking about issues, but not changing. There are also non-process issues. Kanban helps to focus on the system and adaptations.

* Changing technology is easy relative to changing human behavior. If there is no new code, then found that developers would start writing tests, which was a big cultural change.

"MYTH: Kanban has no iterations"

* Sure you can have interations. You have different cadences, and break them up into different cycle times. You can decide how often to have demos, retrospective and planning meetings. In scrum, these happen in predictable times, but the timing of these can happen in more flexible times with Kanban.

* Scrum is theoretically malleable as well, but it seems like a rigid structure. That's because you're told to do all of the scrum practices right first, and then modify it to your needs. 

* Scrum allows you the structure to be able to work together as a team. Kanban can then track how individuals do work and just be a way for individuals to track their work. Sometimes the work isn't well-suited for a cross-functional team -- such as maintenance on a legacy product (which is a myth).

* First virtual Kanban teams were software development teams doing greenfield development, but it also works well for maintenance teams.

* Scrum implementors have a sense of telling the developers that their process is wrong, and they're not doing it right. 

* If a developer team is saying, "We're not getting anything done in a 2-week sprint, can't we try Kanban." It's not about Kanban, and it's not a silver bullet. 

* Developers desire Kanban so that they can be an individual contributor, and use Kanban to kick over to next column and department. But are are they specialists? Or do they not want to do specific type of work?

* Example of having a team of 5 with a WIP limit of 15. Specialists who have the mentality of not my problem. If you don't have a square to move, then you'll be fired. Simplified it, and then determine the next thing. Need to have someone to step up. Don't know how to do it? Need some training? No... I can do it. If you lower the WIP, then people will step up to do the work.

* Can have both scrum and Kanban. Kanban is about visualizing your work. 

* There's a lot of non-software teams using Kanban, and the concepts can be easier

* If you visualize the whole value stream, then you can see the overall cycle time. It provides evidence for the team performance based upon individual bottlenecks.

"MYTH: You should only ever have 3 columns"

* To-Do, Doing, Done are the essential columns. There is resistance to the expanding the number of columns to avoid people not working as a team. But if the cycle time is huge on Doing and there are ways to break it down, then it may make sense to expand the number of Doing columns.

* Track the flow of value through the system whatever the unit is. Use a pull system and not a push system. You can mix things of different sizes, but know which ones your tracking in your statistics. People use Lean Kit for visual management.

Personal Kanban has only a few columns including: Committed tasks, Would like to do, Doing, Delegated to others.

"MYTH: You can't have different stakeholders provide input with Kanban"

* Source of demand as inputs: Marketing, finance and product development. Put a limit on the input for how much can go in. Have a queue replenishment meeting every couple of weeks. Not forced amount of output. There are three spots as input, and let the different external stakeholders negotiate the priority and ordering.

* If WIP is 8, then can have a 3 and 5, but don't recommend that. Lean Kit developers try to do things in 2-week chunks. 

* Percentage allocation doesn't work. There's only number one first priority slot. Try the buy a feature game. The different stakeholders can have a different amount of money allocated to spend depending on the prioritization.

* Why this team decided to use Kanban was that there 14 developers 6-7 product managers. Before Kanban, then there was a lot of siloing, and behind-the-scenes negotiating. Max of 6 to min of 2. Have a queue filling meeting, but no one could decide what to work on. Kanban forced the issues. Developers shouldn't decide the priorities, then the product manager should be doing that.

* Need portfolio development and discussions at the higher level. Pet projects don't always turn into revenue. 

* Big topic in Kanban community is using Kanban at the portfolio management level

  "How to manage everything at all of the levels without software?"

* Had a big room with lots of posters and cards. Lots of duplication for the different levels. Record keeping for Kanban doesn't have to be a nightmare. You can do things on the cards themselves. Wait until the card is done, and then record it to Excel.

OTHER MYTHS OF KANBAN

There's no need to estimate with Kanban. Can't be done.

How can you plan with Kanban?

Can't manage releases with Kanban

Kanban replaces your existing process

Kanban works best for maintenance teams

Kanban does not have iterations

Some tasks can't be broken down enough

There is no such thing as a "Kanban Team"

Kanban encourages command and control structure

You can't track velocity when you use Kanban

Kanban board is the same as a Scrum Card Wall

You don't need WIP if your disciplined enough

Kanban is advanced

Kanban is not agile

Kanban is easier than scrum

Kanban vs. Scrum

Kanban is one of the many agile methods

Kanban requires all work items to be the same

How do I do demos right?

Kanban doesn't work very will with backlogs

Can you do electronic Kanban? Yes.

Kanban is good for managing irregular-sized tasks

[kentbye-blog]

Getting self-organizing teams to really self-organize #AONW

Notes from Agile Open Northwest:

* Self-organizing teams have three characteristics : 1.) Autonomy. Make their own rules and working agreements. 2.) Cross-fertilization and learn from each and share learnings and help each other. 3.) Self-transcendence, and never settle for good enough and self-improvement. People forget autonomy issue. Here's your team or define roles. Liftoff helps to create shared goals and shared values to figure out how to work on things together. Really good at starting things as an individual, but we're not as good at completing things. To get things across the finish line, then that requires a team. 

* Team building cultivates a sense of helping their friends. Invest in each other, so that you can help other team-mates when they're drowning. You care about people, and care about helping the team get work done. Make sure that the teams aren't changing a lot. Do team-building activities, and have a beer after work. Work, and then play a little bit. Be on the same page. Have the same values. Help each other, and get the work done.

* Express the goals in order to get the team to be intrinsically motivated. The benefit of the work has to be seen, and the employees have to see the benefit of the work. 

* They have to care about the goals as well.

* Can you have a self-motivated and self-organizing team without a common goal? No. You need to have a shared vision that people are excited about. 

* You need to have team norms and team agreements.

* If they're doing a three-week sprint and they're not getting things done. Then go down to a one-week sprint.

* Start pairing and using XP practices. XP practices: Paring, everyone in one room, a visual work board.

* Physical cards have really helped things to congeal. Online tools are not in front of your face. PCI compliance requires reviews, but it's not in their face, and so it doesn't get done. Physical boards helps the team to self-organize.

* Be sure that the communication channels open.

* Going through a cultural change, just getting into agile. Need time to catch onto the agile process. Need a good scrummaster to help the team start to see the team the benefits of estimating and the overhead. Get the scrummaster to get the team to set into the cultural changes. 

* It may take 6 months or so. There's some punitive culture that isn't working. Gather the scrummaster community to convince team to stay off the red list, and blaming and shaming people publicly.

* Engineers bad at estimating. Try to write 1-point stories to get better at estimating. 8-point stories aren't well defined enough.

* What if executive management is requesting estimates need estimates by x date in Quarter 1? Estimates are usually wrong, and the team owns the work sequencing. Agile transition has to have the executive branch understand the agile process.

* Worked on a team where the group could not self-organize. Their solution is that they got laid off, and looking for other solutions.

* Team of individuals: Work off of individual backlogs, and now in a team. Think in terms of team goals rather than the individual backlog. Once your task in the sprint is done. Need to treat it as a team.

* Rewarding and reviews as teams? Individual performance reviews? Or team reviews? 15-30 years, and if the team needs someone to test, then there are issues there. Moving away from individual to being more of a team.

* Issues of not having people put in extra effort or time. How to deal with the 9-5 mentality. But 15-hour days is not a sustainable pace. Do sprint planning. Do estimating, and need to take on the amount of work. Long days are not sustainable, and don't create quality software.

* Self-organizing team is empowered to do the work because the feel passion around the topic.

* Need to have buy-in by the whole team. If a team member isn't really engaged, but there are others who aren't, then what are some other options to raise production?

* If they don't care and they're bored silly, then you need to have a conversation to see what's going to motivate them. What are you passionate about? They may be at the wrong place at the wrong time. Look at Daniel Pink says that people need Autonomy, Mastery and Purpose is what people want: Not just money.  In fact, receiving pay checks can reduce motivation. Auto Deposit disconnects people from the rewards of that performance. There's a good TED talk by Daniel Pink.

* We can't motivate other people, but you can hold them accountable. There are working agreements.

* Need to get to the point where you are so self-empowered that you can vote people off of the team. Deal with the interpersonal issues to a point. Do you have a team that can decide to hire? Do the recruiting and hiring? Done that where the team has equal votes.

* If someone leaves the team, then hire a new person. Instead of filling a seat, then determine what work needs to be done.

* Different goals. Different job assignments. It's not an agile team. Not everyone in the same company is on the same team.

* Co-located teams, there are people who aren't participating fully, then how do you motivate them or deal with that?

* Working on a distributed team, and they come together, and then think as a team. Problem that circumvents the team is their input to their team. Individuals with work load, and how to fix individual team mentality. 

* A team forms around a queue of work. The queue will also choose the team. Can have an individual as a team, but there's no collaboration. There's something team like if there is a queue where multiple people are working to do the work. 100% allocation, and are you working on your tasks is very dysfunctional.

* Can you be the team because you're pulling from the queue from specialized work? It sounds like you're pulling from the larger queue of work. Cross-functional notion of scrum is interesting because it's not that everyone can do everyone's task, but it's just itemizing what needs to get done to get a product that works. Need to have members decide to test instead of doing the next task. Jeff Sutherland says that rather to have the team surfing than working on code than no one has ask them to do that yet.

* Do you need a team lead? Not everyone needs to be a generalist. Use the mentoring for other people. Rather than directing work, then it's reviewing and mentoring other people.

* Self-directed teams. One team is extroverted, and the other team was introverted. They decided to merge the teams, and that went horribly wrong to break up the teams. Needed to break up the team again. The team development has to be organic, and don't do it in a non-organic or forced way.

* Everything flows from the backlog. If it's messy, then you'll have a messy sprint. Messy demo. 

* Do 15-minute sprints, and then have working software afterwards. By the third 15-minute sprint, they were able to get something working. Need to be more agile.

* Got a lot of help from refining their process. Need to groom a story before taking it to a sprint. First week groom the story. Need this product with this feature, and then define the acceptance criteria. Then before the next sprint, then do the grooming before the next sprint.

* Estimates provide context to see differences, and to break down tasks. It's not about the number.

[kentbye-blog]

CERN's Exponentially Growing Infrastructure Managed with Puppet and OpenStack

Let's imagine that your team needs to double the amount of servers for next year. Let's also assume that by 2015 you will need to increase your capacity to 15,000 hypervisors requiring somewhere between 100,000 to 300,000 virtual machines. You have an increasingly brittle infrastructure that has reached it's limits of scaling, and you've been told that you won't be able to increase any of your staff for a while. How would you possibly achieve all of this?

Tim Bell is an IT Manager from CERN who faced this exact situation, and his team decided to avoid using any proprietary solutions and instead adopt an open source toolchain to bootstrap their infrastructure. Puppet, OpenStack, and Foreman are at the center of their chosen solution, and these open source tools are helping scientists to make cutting-edge scientific discoveries by managing the infrastructure for some incredibly challenging data collection scenarios.

CERN is the European laboratory for particle physics that is famous for creating the Large Hadron Collider (LHC), which is a 17-mile ring that is buried 100 meters underneath the ground. One of it's main tasks has been is to create the conditions to observe the Higgs Boson particle and provide further evidence for the Standard Model of physics. CERN scientists are also investigating the open questions of:

Why do particles have mass?

What is 96% of the universe made out of?

And what is the nature of matter right after the Big Bang?

The LHC is one of the largest engineering feats of mankind, and it produces an equally impressive amount of scientific data. In order to detect evidence of the Higgs Boson, there are thousands of bunches of protons that are traveling around the collider at a rate of 11,000 times per second. Each bunch contains around 100 billion protons, which produces around 600 million collisions per second. In order to capture these collisions, there are four sensors that are equivalent to a 100 Megapixel camera that is able to take around 40 million pictures per second. This yields a total of around 1 Petabyte of data per second that needs to be filtered down by software to a more manageable stream of 5 to 25GB of data per second that can be captured to tape.

CERN is responsible for capturing and storing this data, and then making copies available to other Tier 1 storage centers. The Tier 2 computing grid then analyzes the data by providing 100,000 CPU days and executing 1 million jobs on an average day. The CERN IT department is tasked with capturing around 25 Petabytes of experimental data per year, and then storing it for up to 20 years. The data center has over 800 racks with nearly 12,000 servers, over 15,000 processors and 64,000 cores. 

Bell realized that even though CERN is on the cutting-edge of scientific research, they were no longer on the leading edge in terms of computer capacity. As Luke Kanies, CEO of Puppet Labs, has said, "If your infrastructure is special, you're doing it wrong." Bell and his team had come to this same conclusion and realized that it was time to look at what other people in the industry are doing rather than continuing to write and manage their own configuration management tools.

At PuppetConf 2012, Bell shared a diagram showing the building blocks of their open source infrastructure management toolchain, which includes Puppet, OpenStack and Foreman:

Bell talked about how OpenStack is very flexible and configurable, but that those options can be complex and somewhat overwhelming to set up out of the box. However, the Puppet Forge OpenStack module has made this process a lot easier to initially set up and reproduce the results.

Another benefit to moving to this open source toolchain is that training has moved from an extended 2-month process of one-on-one mentorship to a much quicker training process. Instead of waiting months before someone can be productive, it's now a matter of days after reading external training books or immediately if they've had previous experience with using Puppet. CERN also has many short-term contract employees that are returning home with marketable Puppet skills that are more transferable to other professional contexts as well as with other CERN-connected, computing grid locations.

Bell also commented that CERN has aggressive growth goals that they'd never be able to tackle on their own, especially without being able to increase their IT support staff. They are committed to contributing back to Puppet and to Puppet forge and he encouraged others to do the same. Not only does this participation help the Puppet project, but it is literally helping scientists to find the 96% of the Universe that we're missing.

You can watch Tim Bell's full presentation here:

You can also sift through his slide deck here:

Accelerating science with Puppet from Tim Bell

[kentbye-blog]

Takeaways from the Agile Testing Open Northwest Open Space #ATONW

Takeaways from the Agile Testing Open Northwest Open Space #ATONW

Take aways from the what people are going to take away from this open space gathering on Agile Testing Open Northwest

Here's the 5 sessions that I liveblogged throughout the day:

Regression Testing as an Agile activity?

Transitioning from Manual to Automated Testing

A brainstorm of tools and frameworks for testing software

Strategies for Managing and Maintaining Automated Tests

Career Development for Testing and Quality Assurance

Also be sure to see Michael Larsen's extensive liveblog notes at his TestHead blog here.

Wanted to find a Silver Bullet, but didn't find one. But we're not alone in these issues and struggles that we're facing.

Is agile dead? It hasn't been baked yet. Still learning about what it can be.

The balance between effort between development and quality assurance on tests. What are unit tests. Automated tests aren't just pushing the button, they're Batman utility belts. Demand from management for more automated tests, and pushback on asking them why. 

Building teams. What types of things do you want to do in the interview process? His approach made sense to other people, and he was on target.

Scrummaster with a development background. What it means to be a tester? Hearing the stories about how they've grown their career, and take that knowledge and better advise others with that information.

Move beyond agile testing? Anything that we can distinctly call agile testing? 8 years in, and it's something that still hasn't fully been answered.

Learned the "Scrummerfal" session.

We learn in the direction in which we ask questions.

Acceptance tests are out of control. A problem that other people may have. A good conversation for when should run tests, how often, manage your test suites. Automated tests are code too, and need to be treated as such.

Session on reviving the notion of patterns on the web. Telling the story of how that might work. Got Linda started within the federation of patterns. If things are shared easily, then they have to be formatted in a way that conforms to some rules. New kind of wiki. It's been in development for a year, and ready to have it start propagated.

Getting a handle around regression testing. Have a legacy testing system, and want to get it more in a more agile way. Know four different approaches from a mailing list, and extended to 7-8 approaches as well as the consequences of those strategies.

Testing qualities and how do you do that in an agile way. Interesting rambling talk that it's not dealt with at all. Here's some approaches to make them more obvious and tracking over time. Perhaps do it in production by monitoring things. "A sufficient way of monitoring things in production is as efficient in production" Perhaps the role of QA is to get the people who are going to write those tests and get the whole team involved.

Thinking tips for testing - Examining the thinking tip and testing the tip, and seen from every single viewpoint.  Slow and fast neural pathways in the brain. The first fast response may be very good, and wait 10 minutes to restore rationality.

Summary of sessions: I am crazy. It is me. But it's also not just me. Others are dealing with same issue. There are systemic and cross-organizational issues, and has provided ammo for him to address the organizational dysfunction. Is the role of dev or QA for automated tests.

Attended a lot of sessions about automated testing. Lots of peoples struggling about the same small things. It's really nice to have a space like this to find out. With my insight, I have a perspective to see commonalities and could provide insights as soon as he has those insights.

Be prepared to be surprised. Understanding isn't always there, and they are in this together. Devs and QAs have different approaches and not working together is destructive. QA are the breaks to devs going faster, and go beyond that and more in collaboration.

Who's responsibility is it? Theirs? Or Ours? Surprised how much Us and Them mentality there still is in a lot of different organizations. This dynamic can get better if you put them physically together. Break down the barriers both literally and figuratively.

Trend: QA talks to QA manager, QA manger talks to dev manager, dev manager talks to developers, and need to have more direct communication.

There are systemic reasons why builders and critiquers work differently.

Agile Open Northwest is another open space conference to keep think about attending here in Portland on Feb 6-8, 2013 -- http://www.agileopennorthwest.com/2012/index.php

[kentbye-blog]

Career Development for Testing and Quality Assurance #ATONW

The QA team and testers are in charge of their own career development, and so what does it mean for someone in QA to have career development?

What have people's career paths looked like?

Started in software quality assurance in 2007, and entered into it laterally. First QA for their company and built out a team and helped to define those career paths. Going through a transitional period of people leaving because they don't see a long-term career trajectory for themselves and what it looks like further on down the road after 10-15 years. Do you become a quality manager? Where do you want to go? Would you like to do consulting?

Conferences like Star East, Star West, and PNWSQ. There's some good books and blogs,

Intro to testing online lectures for free Cem Kaner -- http://www.testingeducation.org/BBST/ Free "Black Box Software Testing" training course

Read a couple of chapters from Code Complete book -- http://www.amazon.com/Code-Complete-Practical-Handbook-Construction/dp/0735619670

James Eisenhauer's Selenium User's Group -- http://www.meetup.com/pdx-se/

Outside of the day-to-day work? How to people work while doing the job at the work.

What are the day-to-day tasks, and how do you progress through various levels? 

Individual contributor who is getting an understanding of tools and methodologies. Responsible for a small set of contained functional testing tasks. Does the function exist as it's been designed to describe. Not a large responsible.

Analytical and technical skills improved. More analysis and investigation of the gaps of the functional requirements documents, gaps in wireframes and prototypes and design deck and all aspects that someone would create. Seeing what's missing. Do anything QA1 does, but a lead on a small to medium-sized project with a lot more complexity.

Identify the gaps, and start making inferences between two pieces, but also the missing pieces. Focus on the business and analytical role. Discovery and prototyping with some testing component, but more analytical skill set. If you're highly technical, then highly robust tools and framework development.

What skills do you need to develop to become a more mature tester?

Cross-Disciplinary and outside of the technical work, and drives to questions around how we fall into low-quality thinking. Look at psychology studies on cognitive biases, and look at propositional logic to see if there is a logical fallacy and use logical tools to evaluate those thoughts and to explain why.

The Selective Attention test, and how we don't see the things that we weren't looking for. Record a lot of the video testing and go back and see the things that he didn't see, and see the things that he missed the first time around like an error message or a slow response. And pair with other QA team member to help evaluate that.

Make a mental discipline of paying attention and keeping your head up. Has a piece of equipment of a Venn scanner as part of the development, and it was dropped and it busted open. Take a look to see what happens when it's put together, and the battery popped out and it resets. May have not noticed that if he wasn't paying attention.

Read the book of "Introduction to General Systems thinking" - Jerry Weinberg - http://www.amazon.com/Introduction-General-Systems-Thinking-Anniversary/dp/0932633498

"The Art of Systems Thinking" is another good book - http://www.amazon.com/The-Art-Systems-Thinking-Creativity/dp/0722534426

Have a specific approach of a mindset, sort out where you're going. If someone doesn't know what their career path is for mid to higher-level QA, then a process of what can they make of it. Either they'll have to forge it on their own as opposed to follow what others have already done.

Fixed vs. Agile mindsets. Need to have agile, and to be able to quickly learn something, and be willing to take on a challenge. 

Cultivate good communication skills for bug advocacy. Understand how your culture responds to your communication styles.

Was at TAO town hall meeting that had a panel was where is QA going in the future? Go to someone who is your polar opposite in the company, and find the sales and marketing people and see what made them successful.  Alternatively, go to the developers and more tightly integrate with them, and start to write unit tests for PHP.

Active listening and understanding your audience. How to phrase your concern in terms of what they're going to value.

Reading the book of "Getting to Yes" -- http://www.amazon.com/Getting-Yes-Negotiating-Agreement-Without/dp/0143118757

Is there a career path for QA that markets and companies will support?

Highly dependent on the company. In his division, then there's an expectation that you must start coding or you're not growing. Want to have a balance of people. Need different people and different minds to avoid having a monoculture.

Have a diverse mixture of abilities and levels of knowledge of QA in her department, and pulled people in from development who have an engineering background. Pulled in Business Analysis and customer support, and there's a melting pot of skills and abilities. Can't have all engineers and all Blackbox testers because everyone approaches it from different viewpoints and methodologies, and will be better at digging up dirt and bugs with that diversity. And each person will have different paths as they develop their careers. Some going into product management. Some go into engineering, development or training. Depends on what your interest is in where you want to go. From the QA position, it tends to have more variance as to where you go afterwards in a company.   

Have many senior-level QA? One person been there 15 years, and some as short as 3 years. People are happy doing what they're doing. Company encourages people to develop on their own unique career path.

What does the 15-year veteran do that others haven't done. Go-to person for anything on the legacy code. Will train newer QA, support and trainers. She's a subject matter expert on legacy systems. Testing is her main job still, but she's used as a resource by all of these different groups.

Where are you going in your career? 

This person enjoys the automation aspect of testing, and developing her engineering skills. First one to pair with the developers in writing the automated tests. Need to have someone who's focused on the engineering pieces of it. Used to be a manager, but doesn't want to be a manager. Wants to get her hands dirty and deal with code, and dealing with people isn't as fun.

This person wants to get more into mobile. Know a lot about browser automation, but want to get more into mobile testing frameworks. Fairly new to quality assurance. Experts in hardware, and didn't make the transition to software, and they didn't have as much as an idea as to what software developers do.

This person has a passion for testing. Has some interest in doing consultancy on testing to help companies maximize their testing outcomes.

Interested in organizational transformation. In the mobile sphere, and there's a lot of interesting approaches for how to deal with agile mobile environment. Looking to find opportunities to help their client figure this space out. Always going to be a next hot thing, and there's opportunities to help sort out how to get through that next hot thing.

Challenges of Organizational Transformation. How to finance change. If focused on getting devices into testing. Mobile can lead companies to develop more agile processes, but it doesn't have to. Agile transformation has a logistics components of when to schedule things. Step away from scripts and start developing critical analytical skills. Never handed a reasonable specification because clients don't understand the mobile space. People still coming from the web browser paradigm of putting everything on the screen. How do you transition that mindset and understand the mobile paradigm and medium. On an Android project with a lot of iOS diehards. Assess what works best. Having a reading list. Having example programs. Understand the navigation paradigm of Android vs. iOS, and examples of what's working and what's not working. Can't get Google Play to feature you if you mimic iOS, you'll confuse the users, and have lots of unintended consequence. Happens in conversations and those take time. Transformation takes time.

Lessons learned in going through an agile transformation?

Doesn't matter as much if you're doing it write. Not have as much documentation. If it works well, then keep doing best practices. A challenge to fit it within the shorter iteration cycles. 

Is there such a thing as agile testing? What contribution does agile provide to the world of testing?

Lisa Krispin & Janet Gregory's of "Agile Testing" book suggests automation + exploratory testing -- http://www.amazon.com/Agile-Testing-Practical-Guide-Testers/dp/0321534468

What tasks you do? How you do them? Rank your tasks. Assign relative value. Document as much as necessary. What's the value of the documentation to maximize the value and minimize the cost.

Using open source as a sandbox to develop and cultivate testing experience?

At her company, used to primarily only Blackbox testing, but now want to have a computer science degree, and almost need to be a developer in order to get into QA any more. Hiring engineers to be QA. It's a trend for headhunters to look for people who have experience in automated testing.

Usually open source has a corporate sponsor, and they hire testers. uTest.com is website where you can get paid to get testing experience.

Just recently hired someone no one with QA experience, and installed Gingerbread software on his iPhone and demonstrated technical aptitude. If they can think about testing in a systemic way, then that's very valuable.

[kentbye-blog]

Strategies for Managing Automated Tests #ATONW

Strategies for Managing & Maintaining A Test Suite #ATONW

Helping customers move to ATTD with either Fitness and Gerkin. It's advantageous at the beginning, but it eventually gets out of control and it's difficult to manage. How to decide to eliminate or not run, and pair down to make it run faster and be more maintainable?

Depends on how you define test.

For people with automated acceptance tests, what are some of the challenges you're facing?

It's like a garden that has some overhead. Can't grow everything all of the time. It gets crowded and overcrowded, and you have to divvy it up. If you have a specific customer test base, then split it up into smaller sections and focus on that section so that it goes faster.

Use a test template. A set of tests that have the parameters for the data. May have 10k permutations. Will not define every test case on a step-by-step, but have archetypes and templates that abstract it out. What does it test? It's regression-type testing, but it could also be acceptance tests.  How long does it take to run? 2-3 hours. If tests are still running overnight, then he knows that he has a problem, then need more templates and permutations.

Be selective about what tests to run depending on what you're doing. Keep them optimized and fast. 2-5 minutes is still a long time if a dev has to wait. Need to optimize for speed. On commit have a CI machine that picks up changes, and then runs the entire suite. If not caught on a local machine, then it'll be caught on the CI machine. How long is too long? Perhaps around 10 minutes, and up to 20-30 minutes. It has logic and integration testing with the database where they have sections that are unit testable, and haven't done that yet.

This person has a CI process all the time and it runs all of the unit tests, and it takes around 45 seconds to run 7000 tests. Under a minute, then devs can wait. They offload the CPU load to distributed computers.

Not stubbing it out and testing the whole thing, and it causes a slow down. WHen talking about acceptance testing on a web app, then Cucumber drives logic through the front-end of a web application, which takes time.  Arguments about record and playback is brittle, and it breaks. So they went with coding tests themselves with Cucumber, but they are running into the same issues with it being brittle and not maintainable.

Selenium IDE can assign XPath selectors, and massaging selectors is easier than writing it from scratch. The IDE can help to manage the test cases.

One of the challenges is that the record tools have a speed advantage in the beginning, but it becomes harder to maintain over time. If they remove a test field, then they will have to change a lot of tests or re-record them if they are playback tests.  Can use config files to abstract out some of the

End-to-end tests are slow and brittle. You can abstract it out, but you will still find that the changes to the logic or database then you tests will break. Why do end-to-end tests? How to get the same confidence and regression testing without using end-to-end testing? You can do it, but it's not easy. Make a conscious effort for doing focus integration testing that have to talk to the outside world and unit tests that don't talk to outside world.

Faster to test it via a REST API rather than using the frontend. Call what the REST API calls, and do lots of mocking. Not good to have to wait 2 hour difference.

Testing and Dialogue with people are different. It's people so just talk to people. Business analysts don't want to look at Cucumber tests.

Alternatives to Acceptance Testing blog from James Shore -- http://www.jamesshore.com/Blog/Alternatives-to-Acceptance-Testing.html

The Problems With Acceptance Testing blog from James Shore -- http://www.jamesshore.com/Blog/The-Problems-With-Acceptance-Testing.html

If designing from scratch, it's easier to do that. But if you're taking on a legacy system, then it becomes harder to get there. It's a long proposition. As you go there, don't let your new code be legacy code. Stuff that exists, then you'll need to have end-to-end testing, which is better than manual testing. You need this to be able to do refactoring. Need tests to do be able to confidently do refactoring, and improve your design and minimize your technical debt. 

How you integrate this into your process? You'll want to know right away as a programmer. If you're not touching a part of the code, then you can do asynchronous testing and multi-stage builds. Use annotations in the test suites to run tests more often and other tests less often, but still gets run eventually. Then change the CI to run a different set of tags. If possible, do it on local dev machines rather than waiting on a CI machine.

Business Analyst or product owner who's used to writing requirements in a document hasn't objected working with the team in making them executable instead of static. Don't know how they feel about it, and it's an improvement on what they used to do. Challenges is to test and touch everything all of the time. If testing entering a zip code that doesn't match the state, then the test will log them in, navigate to product, add forms, and do the entire flow. The next test may have to go through the same series of steps, and it calls out filling out form fields with specific values.

Why not have a login module, customer field module and an address module so that it's small chunks of code that it's own piece so that you can string the modules together with glue code instead of having large monolithic sections. With Cucumber, those steps are pretty efficient in the existing language. The incidental details don't belong in the test because it's not what you're testing.  One test at the top that identifies everything on the screen, and then other should just focus on the given, when, then statements. Chaining steps and validations at each step is becoming fairly brittle.

If want to validate everything along the way, give the states along the way a name to identify with hierarchical form with assertions at each step. Only see results if something has failed.  Why run all of those additional assertions in that test if it's already done already? Sometimes path or data bring in specific unique behavior. Give a way to assign a priority so that they can ignore those assertions at each step along the way.

There's the time to run them, but there's also time to determine what happened if it goes wrong. If it's chained together, then it's faster at runtime, but takes a lot more time to debug if it goes wrong. If it fails often enough, then don't combine them. If it doesn't, then it can be better to combine them.

You'll have a series of stories you're working on, and a set of specific tests for that story. Every test maps back to a story, and it can be hard to maintain. Once in production, then move to a specific tests of area of functionality and combine them so that it runs more efficiently. Use version control, then they go back and look at it. More valuable for it to run faster and be in a better place for maintenance

How often do you go back and review and verify your tests to help minimize duplication? Do you deprecate tests at any point? Two reasons. 1.) If it breaks, then look at it. 2.) If you're working on a function that is related to it, then it's an opportunity to take a look at it.  Currently not looking at tests otherwise. Only looking at it if it's touching the area of functionality about to be changed.

If you're reskinning the GUI, then it's a good time to prune tests. There will be time allotted because it's a lot of work.

If the test is taking too long to run, then it's another time to review and prune.

Are there tools that track code and unit level tests, and then set dependencies to other accpectance tests that need to be run? 

IntelliJ (http://www.jetbrains.com/idea/) and some IDEs have that build in. Can click on a method, right click, and see the unit tests that are connected to it. TestComplete may have something like that, but haven't used it. May need to integrate everything in with TestComplete.

Do run automated tests on all environments? Only run on dev, but not on staging or on production because the operations team allow it. Have to do manual testing on staging or live. Others have experience? More common that we'd like and it's somewhat laughable. Staging would make more sense than production. There's a fear that automated tests could potentially corrupt the data, and if does that, then it takes a lot of time to restore the database. Easier to spin up a dev environment than it is to restore the data if other people are going to be using it. 

If you need to keep your data clean, then use database transactions or use database snapshots and do a rollback. You can use DbUnit to automate some of that.

Run Unit tests in the QA environment and do code coverage, and then have a conversation about where there are holes in their code coverage. See how to do QA tests more effectively from that collaboration. QA and dev will have more conversations coming from the results of code coverage and seeing how to test untested code.

Fragile integration tests that need to be refactored. Challenge to find time to refactor the integration tests. For fragile integration tests, the solution might be to stop doing the integration tests. But you still may be getting value from them.  Brownfield environment and it provides good code coverage.  What's your strategy over time? Profile with unit tests?  As they're redoing units of the project, then they take a look at refactoring.  Go back and forth on what's the right way to arrange data. Putting XML data in a class, but it's run on a VM vs. dev vs. local, and there's some fragility there. There might be a better way to do it than using XML files. Solutions for something better? 

Using XML for a complex object that has been serialized to avoid touching the database, and trying to test the impacts on the code form the API to the rest of the code. XStream library for Java (http://xstream.codehaus.org/) helps with this. .NET has a serializer mechanism that's easy to use. Freezes a class in time.

There's a Data Builder Pattern (http://nat.truemesh.com/archives/000714.html) that people use that creates reasonable defaults, and you can chain together parameters and builds out the class. If you change a constructor, then it'll be refactored along the way. Object Mother pattern is the old pattern.

Other challenges of having a test suite of it being out of control?

A test suite has become a religious thing that we MUST run this test even though it never finds anything. Delay a deploy for a week for running a test that never finds anything. Is it maintained a lot? Just run because people like to see a lot of good green dots. Developers are refactoring a lot of code all of the time, but devs don't build in the time to grooming the automated tests and maintain them. 

Automation tests should be DRY, and should be treated with the same respect as production code. They're protecting you. 

Business Analysts like to have traceability of requirements down to the test plan, but the traceability can disappear for the analysts once it reaches the code level. How to deal with this? Currently using Contour. Don't want to eliminate your manual test case and maintain those, and have it as a backup to the automated test.

Atlassian's JIRA can make the links between these. Bamboo build management tool can also help with this. Like FitNesse for keeping those requirements in a human readable format. Prefers FitNesse over Cucumber, and there are new fixtures that have Gherkin compatibility.

[kentbye-blog]

Tools for Testing Software #ATONW

A brainstorming session on Tools for Testing Software from the Agile Testion Open Northwest:

Mocking Frameworks

Rhino Mocks -- "dynamic mock object framework for the .Net platform"" -- http://hibernatingrhinos.com/open-source/rhino-mocks

moq -- "The simplest mocking library for .NET and Silverlight" -- http://code.google.com/p/moq/

PowerMockup -- "Create Website and Software Wireframes in PowerPoint" -- http://www.powermockup.com/

Apache Maven Test Plug-in -- "This plug-in provides your basic JUnit testing facilities"

JUnit -- "A unit testing framework for the Java programming language. " -- http://www.junit.org/

TestNG -- "TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use" -- http://testng.org/doc/index.html

Acceptance Test Frameworks

Cucumber -- "Behavior-Driven Development framework that lets teams describe how software should behave in plain text." -- http://cukes.info/

FitNesse -- "lightweight, open-source framework that makes it easy for software teams to collaboratively define Acceptance Tests" -- http://fitnesse.org/

soapUI -- "free and open source cross-platform Functional Testing solution" -- http://www.soapui.org/

Browser Automation tools

Selenium -- "a suite of tools to automate web browsers across many platforms" -- http://seleniumhq.org/

Wotir -- "an open-source family of Ruby libraries for automating web browsers" -- http://watir.com/

WotiN -- "similar kind of Web Application Testing possible for the .Net languages" -- http://watin.org/ 

Wotij (?)

Issue Tracking

Serena Mashup -- http://www.serena.com/

Rally -- http://www.rallydev.com/

Contour -- "Contour is a collaborative requirements management tool designed for distributed teams and large enterprise organizations" -- http://www.jamasoftware.com/contour/agile.php

Pivotal Tracker -- "Collaborative, lightweight agile project management tool" -- http://www.pivotaltracker.com/

Trello -- "Your entire project, in a single glance." -- https://trello.com/

Code Coverage

Cobertura -- "Cobertura is a Java code coverage analysis tool" -- http://cobertura.sourceforge.net/

SpecFlow -- "An acceptance test driven development and behavior driven development for .NET." -- http://www.specflow.org/specflownew/

Apache JMeter -- "Graphical server performance testing tool, for both static and dynamic resources" -- http://jmeter.apache.org/

Groovy -- "simplifies testing by supporting unit testing and mocking out-of-the-box" -- http://groovy.codehaus.org/

NCrunch for Visual Studio -- "NCrunch is an automated parallel continuous testing tool for Visual Studio .NET" -- http://www.ncrunch.net/

NUnit -- "NUnit is a unit-testing framework for all .Net languages" -- http://www.nunit.org/

DbUnit -- "DbUnit is a JUnit extension (also usable with Ant) targeted at database-driven projects" -- http://www.dbunit.org/

Firebug -- http://getfirebug.com/

Wireshark -- "Wireshark is a network protocol analyzer for Unix and Windows" -- http://www.wireshark.org/

AutoIt -- "AutoIt v3 is a freeware BASIC-like scripting language designed for automating the Windows GUI and general scripting" -- http://www.autoitscript.com/site/autoit/

MStest.exe -- "MSTest.exe is the command-line command that is used to run tests" -- http://msdn.microsoft.com/en-us/library/ms182489(v=vs.80).aspx

iOS

iCuke -- "Integration testing for iPhone applications with Cucumber" -- https://github.com/unboxed/icuke

FronR (?)

Android

Robotium -- "Robotium is a test framework created to make it easy to write powerful and robust automatic black-box test cases for Android applications" -- http://code.google.com/p/robotium/

[kentbye-blog]

Transitioning from Manual to Automated Testing #ATONW

Notes from a session at Agile Testing Open Northwest about Transitioning from Manual to Automated Testing 

David Whitlock is a Scrummaster at Tripwire. Started as a dev. Dev and QA was separated with remote testers in Vietnam. Now testers and developers sit next to each other. All co-located in Portland. Went from 5 distributed testers to 2 testers per team. Transitioning from changing testing mindset from manual testing to automated testing.

What are some experiences that people have from transitioning form manual to automated testing?

Moving towards test-driven develop to building in testing, but still have to resort to manual testing. Release day is a struggle because there are a lot of smoke and manual tests. They can't afford that manual testing time any more.

How do you bring automating into an agile environment?

1-week iterations, and every story is tested into the story. Write test first, and then write the code. Have unit tests, but also have automated acceptance tests. Feel good enough with test coverage that don't need to have an manual testing. Some exploratory testing to discover blind spots.

QA to have dive into tests that they want to? Yes. Do pairing. Do the tests drive a GUI? Yes, use jUnit to drive the GUI. Wrote own framework to test the GUI. Running the tests get really slow 2 years out. They've mocked some of the tests, and just testing under the GUI. Have end-to-end, "customer" tests, but it's a smaller percentage.

How to get to automated tests? Two week iterations with no automated tests is a big problem. How to shift the culture and get people to own everything?

Are developers doing test-driven development? Perhaps move all QA people off of that team. Let the smart developers figure out how to write a test, and they have to become their own safety net. Primarily due to budget, but they were very successful and delivered software. What if they only write tests to pass? Have to define done, but at the end of the day they needed to deliver a coded, tested and documented solution.

Do a flow map to find the biggest bottlenecks of taking the most effort and time and doing the biggest impact thing first.

Mine bug database to figure out: 

* What changes the most often? 

* Where are the biggest bugs and the most frequent? 

* What's taking the most time? 

And then write automated tests towards those issues.

If doing no test automation, then what type of test automation is there?

Start with Test-Driven Development because it's so successful so often. Helps to guide their code architecture, and you get better code out and faster.

If you don't have the highest-level of buy-in for time to test, then it can be hard to do. But someone else says that if a dev doesn't want to test their code, then he'd go and hire a new developer. Clean Code book that talks about that conversation. http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882 Team will need to value automated tests.

Don't ask for permission. If it works, then I get credit. Then if it fails, then it's the manager's fault for approving it.

There's unit tests, but what about the other automated tests?

If the automation team is separate, then someone to that team. Scrummaster needs to allocate time for it.

Do you have a dedicated automation team? How does that work? 

They use python. Team will ask for a task to be automated. Then it'll go to the teams after the smoke and automated test, and then it's sent to the team. Automation team is overwhelmed because there are 12 agile teams and only one automation team that is served to them. Need to transfer this functionality to the actual scrum teams.

Lots of companies in a transitional phase where QA is separate from development. It doesn't work very well for them to be separate. The two have to be the same. The developers are the automation team. When they're doing the test part of the development, ten the developers are pairing with the automation team. But how does that fit within 2 weeks? Do less features, and it's test-driven and so it comes first. Automation is part of the done criteria for the story.

To get test automation, you may need to go at a lower velocity with less features, but an increased quality.

Need to separate unit test and automation. Unit test and automation became the same thing, but it's not true. Automation is a collaboration of dev + QA working together in building a framework or set of tests to test a feature. Start with an easy regression test? Do a log in. Write a module and get the ball rolling, and it's slow if there are small sprints, but eventually you'll build up your test suite.

Is your application able to be automated? Can you support and maintain the testing framework over the time?  Write 10k unit tests vs 100 system level tests? It may not discover any bugs.

Once an automated test is written, then it'll likely never catch a bug. But it gets worse. Management sees the green lights, and they think that the program is working, but that's not always true. You need to have someone look at the test in order to see if it's actually passed and it's working. But it doesn't mean that it's shippable. 

That depends upon how the test is written though. Automation can be written so that it's lazy and not smart, but need to write it so that it can pass and it can fail. Need more broad verification. What's the goal of the automation?

Have the same problem with manual tests. If it's not a good suite of tests, then they'll say that the test is working. So what makes a good suite of tests? Have well-defined acceptance criteria. Make it a team decision, and not leave it up just to the QA team, but also have to look at the costs of automation. It might not be if it'll take 8 weeks to automated. It might only take 10 minutes to manually test. It may be 10 years of 

Mandate is to make regression tests faster. Take a long time, but if a test doesn't take a lot of cognitive activity, then it makes sense to automate those. See how many stories would it take to automate. Can eliminate regression testing within a number of stories over a number of sprints, and present that to the business analysts and product owners.

QA needs different layers with a good mix of different approaches at different layers.  Separated GUI from the business logic to test the business logic, more stable, faster to write and less brittle. It's the layer of Shangri-La.  GUI is really thin, and have limited number of transformations for the JavaScript. Have exploratory testing focus on the UI. Manual test the web GUI, but test the backend with automation.

How to understand requirements and test different types of data? What is the field supposed to do? Then start to break it with bad data. Possible to call info from a config file? Someone has used census data to harvest random data to do input validation. But if are doing customer validation, then can also use some subset of production data. Random data has limited value if you're trying to test the customer experience.

What about the test environments? How real do you need to get? 

Physical machines vs. VMs? Can a web service mock some of it out? How real do you need to get? If doing a functional automation test, then do it on a MacBook Pro -- But wouldn't want to do a load test. What's the target of the automation? There are different levels. Also depends on how distributed your environment is, then the testing then the test environment needs to mimic the production environment. If you need to go fast, then do you do a VM or do you do it in the staging environment?

Can only go as fast as the slowest person. Automation is the slowest part, and so you need to have team participation to get it go faster.

What automate? To go faster, but also freeing up resources to do exploratory testing and to innovate.

How to sell value of automation? Involve the feature team and automation in the conversation. Tear down the walls when possible. Hire for automation skills -- or alternatively train them to do it. They may not have the skills, but they're trainable. 

How do you train the automation? Pair programming is an option. See what you want to automate, and then allot some time in a sprint to do it. Online sources are SQA Forums (http://www.sqaforums.com/ubbthreads.php), TestComplete (http://smartbear.com/products/qa-tools/automated-testing-tools)

Favorite books - ANT book. Any of the O'Reilly books on build languages. Hard to find a good test automation book, there are a lot of good blogs and online resources there.

Need a balance of skills on a team. There is someone who wants to geek out with testing, and give them time to experiment.

No one has tested the tests. Automated tests are the least amount of tests. Some tests have passed when they shouldn't have. Has anyone tested the tests or test the automation? It's a programming thing where you need to have reviews. It shouldn't disappear into a framework, and it needs to have a review process. You can still review it even though it's automated.  Automation will die without that review process. 

When introducing automation, there's the instinct to build it from the ground up. Don't reinvent the wheel. There are a lot of frameworks and tools out there. 

[kentbye-blog]

Regression Testing as an Agile activity?

Notes from a session on “Regression Testing as an Agile activity?” at the Agile Testing Open Space event on  October 11th.

This specific team has a product, and needs product testing, and it takes a month, week or days. Scrum and XP do TDD, and then release once a week or release once a day and continuously. Releasing more often is better. What about testing?

Have seen some companies suppress integration test cycle and some have eliminated.

With a mature agile process. Great feature test within sprint. Develop a feature set and it's tested. Not building on functionality test, and have a more difficult time. Net new development is easier.  Legacy development without a set of regression test. How to prevent waterfall event at the end.  Still do waterfall releases on a bi-yearly or quarterly cycle. Regression cycle at the end is familiar and is painful. There's a

N+1 for feature-based testing, and do that without balloon payment at the end.

Full integration testing end-to-end, 4 people a solid week. If it's buggy takes longer. Depends on the tester's knowledge of the system. Skilled testers, but unfamiliar with the legacy system.

File bugs, and then wait for a new build for new testing. Found a blocking bug, and then wait. Percentage of waiting? Low wait cycles. Turnaround is less than a day, and devs swarm. Test plan is housed in an excel spreadsheet, and not know if everything is caught. Not automated.  How to get through the testing phase in a more graceful system.

Assumption that area that's being testing, then people are familiar. Testers self-select what area they will test. Is sounds like "Session-based test management" is similar to what you're doing, but has more tools.

Need automated tests in order to release continuously. Possible to do UI tests, but it takes.

Dev and QA are tightly coupled during the sprint. But do the regression

Include regression time with feature function dev time during the sprint? With a weeklong time sprint, there’s not a lot of time.  Need to have time allocated for quality. If you have a stabilization sprint, then it’s a sign that things are not going well. Turning into regression cycle. Account for the time to determine whether you are done.

Make a story, and if passes acceptance criteria, and their regression testing is built into continuous integration. Have additional regression testing, and run every time build is deployed to demo server, and the tester is the regression test point. Automate as much as you can. If there’s JavaScript that needs manual testing, then use Rapid Reporter, and do manual testing. Minimize those GUI tests, and avoid doing 200 tests.

Definition of done includes not breaking anything.  Schedule time during the sprint and do less testing.

How to eliminate regression tests? If you don’t have a GUI, then it’s easier. If you do have a GUI. In theory, need to be code complete before regression, but in practice you’re not going to have code complete and finding bugs earlier during the cycle is  a reasonable risk model.

Make light and quick Selenium tests, that runs headless and no browser so that it’s quicker. Separate GUI from the backend and have a REST API that can be automated. Do exploratory testing on the GUI in a fast way, but decouple the logic from the GUI.

.NET Ncrunch automatically runs the tests at different layers. Runs tests upfront and set it up to run all tests. Uses this for integration tests at an API level in Visual Testing. Nunits.  Fitness or Watir.

Make regression testing smaller, then need to build quality in sooner within the development process.

Proceed final regression cycles. Have unit tests. Code reviews. High collaboration. Can only do so much from functionality perspective, but don’t have the creativity as a team, but they’re still waiting for the regression cycle. Possible to scale back the features on a sprint and define done as including regression testing.

If GUI is not where the risk is, then go below the GUI and write more tests.

Watson web application testing and “Secret Ninja Cucumber Scrolls”

http://cuke4ninja.com/ for testing for frameworks.

Are other people skeptical of Code coverage metrics? Yes. If 73%, then that’s only that which has been executed. If code coverage is high, then that’s when testers find more interesting and challenging bugs and it’s more fun as a tester.

Stabilization sprints. Not just stabilize system, but see patterns to prevent bugs and how to discover them earlier. Are there certain kinds of bugs or certain kinds of programmer errors?  Get the learning out for helping to build more quality into the system sooner. Build attention to the lesson learns and build mechanisms to find them earlier and not at the end.  What’s the need for the regression sprint? Not discovering bugs soon enough.

Eliminate regression testing if you’re more open to increasing tolerance of risk. Roll back or offer a fix, re-tested and re-deploy. Make re-testing and re-deploy cycles slower. Worry about moving challenges into the hotfix zone. Also think about who is being put in that position of risk.

This is a business decisions. QA makes quality transparent.

 If you have copy and pasted tests, then try to consolidate and optimize your tests.

If you have bloat, then how to reduce if it doesn’t have a business purpose? Make a folder of to-be-deleted, and delete them later. Run some tests less often to see if they’re not telling you any

What Pattern for tagging?

Has a client that has silver smoke tests that run hourly, bronze smoke tests run on every check-in, some tests are run once a day. Which ones do you need to know about sooner? May need to delay knowledge by not running everything all the time.

Smoke test tool – How often do you find a problem with the tool? If not very often, then look at the area that is the most troublesome. Social integration is the area that has the most issues. May not need to run all of the tests every time. Has tests that are primary, secondary or full monty.

[kentbye-blog]

State of the Puppet Community 2012 #PuppetConf

State of the Puppet Community - PuppetConf 2012 from James Turnbull

James Turnbull wrote some books including "Pro Puppet," "Pulling Strings with Puppet: Configuration Management Made Easy," "Hardening Linux," "Pro Nagios 2.0," and "Pro Linux System Administration."

VP Technical Ops at Puppet Labs, education, training and community management. Australian accent. Film fan.

He's amazed that Puppet keeps growing. Puppet gatherings are great for being able to share the stories and to get real value out of their experiences, have great conversations and communicate. 106 employees at Puppet. Wore all of the hats except sales since the beginning. Pleased to have community management under him as well.

Ops tools world is a bit like Aliens. We're kind of like the colonial marines without the massive death toll and implanted parasites. Princess Bride is his favorite film, and will use moments from the film in the presentation.

Introductions

State of the community

Initiatives

Ask what your community can do for you

Questions

Introductions

Dawn Foster is the new community manager, and her 2nd day on the job.

Community lead at Intel. Lead Tizen and Meego, and worked at Jive. Started as a sysadmin back in 1995, and getting back to her roots. CS degree and MBA.

Will be lurking and learning for the first month. Work on publishing some of Puppet's community metrics. It's presented once a year, and wants to publish metrics more frequently.

Can be reached at @geekygirldawn or dawn at puppetlabs dot com

What does Puppet community does really well? What are things that it wants to do? Community Point of escalation. If it's getting ignored, then she can track people down and help get some movement on issues.

Andy Parker is the Puppet Team lead

Did some system administration, and used cfengine. Wrote a paper on configuration management. Does cycling and long-distance cycling. New to open source development, and is learning. Background is financial services software and it was a lot more locked down.

What do you have happen? What do you want to do? How can I help?

Andy at puppet labs and @zaphod42 on IRC

Eric Sorenson, Puppet Platform Platform Owner, and owns the roadmap.

SysAdmin at Apple. eric dot sorenson at pupplabs dot com and eric0. Talk to him about pull requests and features.

Metrics for the Puppet Community

Metrics from 2011

Puppet user mailing list membership was 3000

600 messages a month

#Puppet IRC channel averaged 400 people - One of the busiest IRC channels

Puppet on GitHub had 200 forks and 400 watchers

We had 50 modules on the Forge 

Metrics from 2012

5000 members on the mailing list

1200 a month Doubled the number of messages in a year 

#puppet IRC channel averages 800

Puppet on GitHub has 350 forks and 950 watchers

We had 500 modules on the Forge - How to contribute back features

Turnbull was blown away. Constantly amazed for how the community has organically grown together.

Engineering. What has Puppet Labs done in development? Needs to improve communication of what Puppet Labs is doing to the community, and will see more information.

Number of commits to Puppet per year:

2005 - 475

2006 - 1123

2007 - 1205

2008 - 1205

2009 - 930

2010 - 1254

2011 - 2454

2012 - 2447 (just year to date)

Will going to see a lot more commits in the future.

Ratio to core commuters to contributors has been growing over the years

Up to 99 contributors and 28 core contributors.

Don't have access to all of the platforms, and so there are some contributors who only contribute 1 to 2 commits, which is really helpful to get compatibility on all infrastructure platforms.

SLOC has gone from 21k to 141k since 2005.

What's next for the Puppet Community?

Questions and answers - Talk to Dawn any of your needs

Developing a Q&A Site Not everyone likes mailing list. People like forums, and putting together a QA site in the site of forums and leverage the success of sites like Stackoverflow. Mostly sysadmins, and getting better at design. Emphasize curated on practical questions and practical answers. Get questions indexed by Google instead of looking through layers of mailing lists.

Becoming more responsive

Sometimes we suck at communicating. Being in a start-up, it's easy to be heads down and not have the bandwidth for communication. 

Puppet Labs is the primary sponsor of the open source community, and will foster more interactions in the community.

We're getting better.

There has been some staff movement with the community management position. Mike and Jose did a great job at organizing the Puppet Conf and Puppet camps.

Call us out if Puppet needs to be more responsive or is not doing things that you want. We'll admin mistakes. Mistakes will be made. Honesty and transparency are key to fixing things.

More metrics

Dawn loves metrics. It's to understand who the community is, and what they are. We want to be transparent and accountable.

More awesome

Talk to us, complain to us, give us feedback that we can action and help us out.

State of the community

Puppet needs the community's help as well. Contributing to an open source project is personally and professionally rewarding. Turnbull used to work at a bank, and it was not satisfying. Worked on Puppet Labs in his spare time, and it got him a job. It's significantly changed his career, and he's much happier. Opportunity to give back to a community he cares about.

Needs help with Documentation

Wiki needs updating. Move info from Wiki into docs site: http://docs.puppetlabs.com. Put in pull requests for migrating documentation.

Documentation: http://github.com/puppetlabs/puppet-docs

Puppet Redmine has tickets: http://projects.puppetlabs.com/projects/puppet-docs

It makes people lives better and easy to get involved.

Facter

Have a platform Facter doesn't know about?

Have a resource or infrastructure component that you'd like exposed? 

Then help to expand the test coverage

http://github.com/puppetlabs/factor

http://projects.puppetlabs.com/projects/facter

Modules

Share and publish your modules. Workflow is not right. Bury us in modules and we'll get it right. Make it a pain point by contributing more modules

develop modules

There are module bounties that involves winning something. Graphite and log stash module bounties. These will be published on the blog. We will be featuring a module of a week. Help to write about a module

Forge is at http://forge.puppetlabs.com/

Code and Testing

Triage Puppet bugs and features

It's easier than it looks. Parts of Puppet are still a mystery to James, but things on the edges and extension points are easy to write and integrate with and to pick up Ruby. 

Provide version and reproducible information and write good tickets.

Puppet 3.0 fixes were because they were voted on

Write code, Write tests

Team will help to provide input.

Test releases -- Get as much feedback as we can for each release

Get some tips

Getting started with Contributing to Puppet and Factor

Hailee, Tesca and Ruth will be presenting on how to get started

[kentbye-blog]

Using Puppet at SpaceX #PuppetConf

Joachim "Jok" Thuau, SpaceX talks about how SpaceX uses Puppet.

Linux system engineer, and came from the video game industry. Been at SpaceX for a year. It's a rocket company and first company to send a vehicle to low earth orbit. First to recover the vehicle. Visited the ISS. Designed the Falcon 1, 9 and Heavy rockets. It was too small to do anything and on a bigger scale and build scaled up rocket. And then used 9 engines -- Falcon 9.  Planning Falcon Heavy, and it will be 3 Falcon 9's stuck together. Rocket doesn't have any payload, and using a capsule named Dragon.

Puppet. Talked about Puppet for a long time. Problem: Need to deploy 10s or 100s of machines. Have Puppet, but haven't used it. It'd take hours instead of months. Getting lots of requests to deploy a lot of machines. Signed up for scaled university, and did a 3-hour Puppet training. Convinced them to use Puppet, and just got started. Sat around table for 3 hours and had only seen stories. Installed Puppet and started simple, and it was immediately useful.

What we have

Debian/Ubuntu

Kerberos (windows KDC/Active Directory)

LDAP (windows LDAP/Active Directory)

"Apt" mirror (with debian squeeze/ubuntu LTS)

Tons of scripts!

Deployment

Using existing scripts and replacing them with puppet.

Install packages from our local repo

Push configs with puppet

Using exec (in a couple of places only) to "fix" a couple of things.

This simplified things a lot for us

Where are we at now?

Plug desktop/server in (power/network/keyboard/video)

Boot "off the network" (pxe)

Pick on pxe menu what you want to install


We have a number of "presets" we use

Install starts and asks the user for hostname

Machine uses a pre-seed file to setup stuff initially

Installs puppet as part of that pre-seed from our local mirror

Generate and registers with puppet server

(login on server and sign cert)

Cool stuff to come

Foreman & Puppet server infrastructure overview

The Foreman

Really nice and easy to setup

Has "proxies" for handling stuff with DHCP/DNS/PXE automation (we’re planning on using this, and it’s a little more difficult for us, since that part of the infrastructure is windows based)

Nagios monitoring

Host lists/services/dependencies built by puppet


HPC

We have a compute cluster that we are starting to rebuild using puppet for automation and validation (test cluster with a couple of nodes, validate config, and push to production)

Showed video of Dragon capsule docking with ISS. Showed video of rockets returning.