Student reading in theory and practice

Brain dump(lings) of an Arcadia Project at the University of Cambridge, trying to find out more about how students go about their academic reading.

[camreading]

asdasda

dasdasdasd asddasd

[camreading]

This video presents the outputs of the Arcadia project as it was done for main stakeholders, using the design room as a mini exhibition space for all the design artefacts. I am talking about user centered design research and design, highlighting all the steps within the project, and showing most of the outputs. If you would like to understand this project and its outputs, it best captured here in 1 hour 30 minutes.

[camreading]

I am very happy to announce that the final project deliverables are now ready and can be accessed freely by clicking on the link above. Have fun diving in!

[camreading]

Prize draw - we have our winners!

Very happy to announce that we did our prize draw today morning, and we have our winners. The draw was done at CARET, and witnessed by our lovely administrator Stephanie Saunders.

Here are the winners:

Personal Reading and Book Diaries:

1st prize (choice of Amazon Kindle or same value book tokens)

Caitlin Wylie (History of Science)

2nd prizes (£30 Amazon vouchers)

Alexandra Day (English)

Bethany Megson (Cassics)

Sylvia Christie (English)

Sarah Wandler (ASNC-PhD)

Samantha Mullender (Natural Sciences - Bio)

Online survey:

1st prize (choice of Amazon Kindle or same value book tokens)

Alireza Tabatabaie (Education)

Congratulations to all of you and thanks very much for participating in the project - you are all awesome.

[camreading]

Are we there yet?

Yes - well, almost. Last Friday I presented the initial findings and ideas to Anne Jarvis (UL Librarian), John Naughton (Arcadia project director), John Norman (Head of IT at UL and CARET), and Patricia Killiard (Head fo Electronic Services at UL). The initial response and feedback was very positive, so I am hyper excited by the prospect of setting things into motion and start actually creating some awesome things.

At the moment I am working on creating the final artefacts, to publish them here and share all findings and ideas. Tomorrow I will also arrange the drawing of the prizes, so stay tuned and watch this space! 

[camreading]

Design - not the drawing type

After coming back from the Christmas break, I started the design phase of the project. Actually most of the times I think, there is a bit of a mystery and magic around transforming user research data into usable designs. My hand wanted to grab a pen a draw boxes on paper, thinking of zillions of ideas and features which would help people do things better. But again, my brain disagreed and luckily managed to hold this back, as I felt that I am jumping ahead too quickly, cutting corners.

The first thing I needed to do was to get all the things which I have learnt out of my head. The great people at Head London design agency helped me a lot in this, and together we found an easy and understandable process to start shaping understanding into useful solutions. As a first step we wrote up all of the key findings on post-its and prioritised them on a huge wall. An example of such a post-it would be: "The large group of students have problems accessing required readings". This was an extremely useful activity, which forced me to spell out findings and making sure that I can communicate those to people outside of the project. The next step wast to formulate design goals, based on the prioritised findings. This is actually a hard task to do, since a lot of the times, people would naturally formulate "pseudo" design goals which are actually not design goals at all, but rather an attribute of the system we are trying to create. Very useful to understand the difference, and being able to formulate unique and relevant design problems is crucial. We ended up with 5-6 very good design goals like "How do we help avoid book shortages?", which served as a basis for the next activity: ideation.

The reason behind clear design problems is not only being able to articulate and communicate. They serve as starting points and triggers for ideas, which tries to solve a given design problem. The way we approached was to have a large number of ideas solving each of the design problems, in the form of user journeys, which had to contain an actor, an object and a goal. Expressing ideas this way, almost like simple cartoon strips, encourages designers to think of user flows rather than individual features or functionality. Features are assembled by disseminating, combining and evaluating all these flows. Once features are there, through a prioritisation activity we can start drawing a bird's eye view of the system, and mapping out how different features connect, to best support the design goals and the user journeys. This is when my brain is comfortable with drawing, and this is where finally sketching of individula screens can begin.

Another interesting thing which I managed to get a better understanding in with the help of Head, is the ability to communicate design findings. They showed me a number of different ways to visualise complex relations and complex data. Very useful. I realise that without pictures this is dry and unimpressive, but please bear with me, I will make all this more visual once the design phase is completed and the final artefacts are made which is very close now.

[camreading]

Making sense of things

Since the last post was in December before breaking for Christmas I will try to briefly summarise what happened since I conducted the hangout sessions and the library visits.

I have spent most of the time before the Christmas break on analysing all the data which came in through the interviews, diaries, the focus focus group and the hangouts. This was no easy task, since I have a lot of data. I listened to all of the interviews again, and made notes off all what the students said, following my usual "Life", "Actions", "Discovery", "Organisation", "Social" and "Technology" themes and sorting all the information according to these. Then I looked at all of the diaries, and created a cleaner digital version of each, visually drawing out all of the reading activities, categorising them and counting all of the durations. This also enabled me to create easy to read accounts of reading activities for each diary, with some basic quantifiable data. Since I did 4 hangout sessions, 8 interviews, numerous library visits, and armed with all what I experienced working at a library, I already understood quite a lot about reader behaviour and motivations, but the diary study data addd some depth and quantifiable validation to those assumptions. As a final validation step I organised an online survey which actually just started a couple of days ago, with the view to validate my overall assumptions and reach a wider audience with my questions. Thanks to Google Forms, the data from this can be easily disseminated and I am receiving a good number of free text inputs, describing real life problems and experiences around reading. By the time I went away for Christmas, my head was already full of experiences and understanding and a warm fuzzy feeling of assurance, since I knew all this stuff comes from actual people doing things in real life.

[camreading]

Sitting behind students

Last week (-2 days holiday) and this week I spent organising and conducting "hangout" sessions. A hangout session is spending time with a student, shadowing his/her activities for about half a day and observing how people read and do related activities such as note taking, searching etc... These sessions proved to be one of the most data rich experiences, and I feel I have learnt a tremendous amount, even though I only had 4 sessions so far. The ability to observe in context and have the opportunity to clarify things soon after an action proved to be very valuable and brought up many issues which I am starting to try to consolidate in my head and on paper. I did a bit less interviews and hangout sessions than initially planned, but now I am actually glad that I still have a few incentives available: I can use those at design stage to get some feedback on prototypes and sketches.

As the data from the diaries comes in (which has started now) and as I process all the incoming information so far, I would like to start organising for the validation survey. This is tricky for various reasons: students are mainly at home at this time of the year (although they can participate online) and the analysis phase of the project is closing in shortly next week. My aim is to have a preliminary online survey set up by the end of this week, and with luck publish it next week when I started to make sense of the available data. This way the survey will not be a standalone probe, but would build organically on the top of all the data gathered so far, validating assumptions which are already formulating in my head. The survey probably will run in parallel with the design stage to allow wider student participation.

This week I also plan to visit some departmental libraries to get a sense of variety and understand some of the issues around physical spaces. Another thing which is in progress is organising 2 days in the design phase at a commercial design company, allowing me to bounce off ideas with other user experience and information architecture professionals, which I think will be very valuable - he ability to creatively challenge a piece of work like this is very important I think.

I realise that I haven't been doing very well on keeping up with my weekly whacky posts, the reason is that I simply had no time to think about things like that in the past few weeks. I am hoping now that the observation period is coming to an end this will change.

[camreading]

Smashed by data

Last week was very busy, but extremely productive. My head is full of fresh experiences, and a LOT of data which I am trying to let sink in. I am consciously making an effort not to get bogged down with details at the moment, and keep a fairly high level eye on everything. Here is a short account on what I have been through the last week, adding some thoughts along the way.

Interviews Last week I finished all the planned interviews, managing to speak to 8 students. Luckily I had access to a diverse crowd and the amount of things I managed to uncover is pretty good. My approach here was to have semi-structured discussions, meaning the conversation touched on a number of base topics such as Discovery, Organisation, Actions, Daily life, Technology, Emotions and Social interactions, but was not pre-scripted allowing to organically explore the topics with the participants. Each interview was recorded, so that I will be able to spend more time making notes of them during the analysis phase of the project, and allowing me to spend only a little time after each interview gathering my thoughts and impressions. 

The first things which struck me as I went ahead with the interviews is that students are extremely helpful and accommodating when they see how a piece of research can impact their daily life. For me this yet again proves that the student community is more than willing to engage with activities like this, we just simply don't tend to take advantage of this incredible resource often enough. However I made a mistake when planning the project: the amount of time needed to organise the interviews and the physical location of the interviews meant that my library observation had to be constantly interrupted. I vastly underestimated the lead in and out time around the interviews and the amount of time needed just to maintain email correspondence with students. Next time I know that these two activities can not be done in parallel. 

Focus group Last Thursday (Dec 1) we held a 2 hour long focus group with 5 students. I had 7 students signed up, but 2 had to drop out due to university commitments. Again this proved to be an excellent way of finding out more about experiences, desires and values. The first part of the focus group was spent on moderated discussion around the base topics mentioned above. Then we split into 2 groups and embarked on a mission to construct a robot and fairy which would serve in a library. The activity was really fun, and the general feedback was very positive. We concluded the session with a short discussion on technology and the students sent messages in a bottle for future students on reading. For this activity I asked my colleague Anne-Sophie to help me out with note taking. She used the wall of the room to take notes on post-it notes and organise them in real-time according to the base topics, and differentiating between who said what. We ended up with an almost full wall (5m) of organised notes, and the standard voice recording of the whole session. 

Hangouts On Friday I had my first "hangout" session with a Philosophy/Psychology student at the Trinity library. This was one of the most "data rich" experiences in the project so far, the amount of detail you can capture just spending informal time with a student is astonishing. She gave me a tour of the library and then she went on to do what she would normally do in the library, with me shadowing her in the background and peeking over her shoulder to her screen. During the session she was very helpful and offered clear explanations on why she is doing certain activities. Definitely one of the highlights so far, and one of the best use of the £30 Amazon vouchers so far. 

Library observation Besides all the above, I worked at the English Faculty Library when I had time, and observed what's going on. This was quite fragmented, but still managed to make quite a bit of notes and now I can see some patterns/regulars which wouldn't have seen otherwise.  As I said earlier, next time I will not plan observation and other major activities on the same week as it is impossible to do them well, however I was somewhat forced considering how close the end of term is (was).

This week I will continue with hangouts, and gradually switch over to focusing more on visiting more libraries and doing more observations. At some point I will also need to start thinking about the things I learnt and start to construct an online validation survey for my assumptions. Luckily there are still students here and the library is still not empty...

[camreading]

Personal reading diaries are now available to download.

Click here to download it

The aim of a personal reading diary is to capture a student's reading, and reading related activities with some context. The diary should be kept for 7 days, then sent back before 19th of December 2011. Each line in the diary should be filled with one reading event. A reading event is any activity you do, which involves academic reading, or related activities such as time at the library, searching for books, reading online journals, constructing a reading list with a supervisor regardless of its place or whether it is online or offline. 

Everybody who keeps and returns a personal reading diary will be entered into a draw for an Amazon Kindle e-reader or the same value in book tokens. As an additional prize, 5 £30 Amazon voucher will also be drawn out among participants.

[camreading]

The downloadable book diary form is now available.

Click here to download it

Book diaries follow the life of a book whilst out on loan. The idea is that a borrower who borrows a book from a library would record the activities in this diary related to a specific book, even if the book gets passed on to another person, who would need to continue the diary. The aim is to capture contextual information on when, how, why, and who uses the book - so the diary should be kept alongside the book as much as possible. The ideal time for this would be on 2 day to 2 week loans, and the diaries should be returned by 19 of December, 2011.

Everybody who keeps and returns a book diary will be entered into a draw for an Amazon Kindle e-reader or the same value in book tokens. As an additional prize, 5 £30 Amazon voucher will also be drawn out among participants.

[camreading]

Hard planning

I've been pretty quiet in the last couple of days, since things started to speed up, and soft planning of the previous week turned into more focused preparation with more tangible outcomes. Since things are progressing on multiple fronts here is a little rundown of each individual pursuit:

Where to start? What to ask first? Substance vs. method My main takeaway from the 1st planning week was that whilst I started to feel immersed more and more in the world of student reading, and I felt that I did quite a bit of background reading, I simply still didn't know what to ask from students. I felt that I am ahead with methodology and I have a clear picture of how I want to do things and why, but I still missed the substance, the core set of questions that I would like to ask and know more about. This didn't necessarily meant that I couldn't think of questions to ask, on the contrary: I had too many of them, covering too many topics. After thinking about it for a while I had concluded that I needed a higher level approach than individual questions, and so themes were born. I wrote down all the initial questions that I could think of, getting them out as quickly as I could and grouped them according to what would be the most appropriate method to ask it. After this exercise I ended up with a long list of questions organsed into several tracks. I grouped the similar ones (with sort of an affinity mapping, but quicker/simpler) and identified a couple of themes which seemed to run across multiple questions. Here are my themes:

Everyday life

Discovery

Actions / Activities

Environment

Organisation

Social interactions

Technology

Emotions

I will be using these themes to guide and serve as lens to look at student reading as an experience. All enquiries will touch on these themes as a baseline, and will look for human factors within the everyday life of students such as needs, behaviours, motivations, experiences, values, and desires. Instead of focusing on particular preconceived questions, I will use these themes to guide interviews, run through diaries, and set the framework form my focus group. 

This fitted well with my other endeavour, which was to leave space for serendipitous exploration, and at most only semi structure things. I didn't want to script or plan all aspects of this research, I think the most interesting things come from unexpected places, and this type of approach models real life better I think.

At this point I was happy with the substance. I felt I had good focus points and I was confident that I could explore each individual theme in greater detail, with the exciting view that the road within each theme will most likely lead me to interesting places and situations. Oh, and I am expecting that the best fun will start when the connections between these things will start to surface. (if ever, but hoping)

Recruitment

As I mentioned previously I was very pleasantly surprised with the student response so far. I have close to 300 quite diverse students, who are willing to help out, and most of them are prepared to sacrifice more time than an online survey. I am hoping this has to do more with the genuine interest in the topic than the incentives, (which are pretty good I think) but either way it shows that students are willing to contribute and participate in formulating thinking and design within the university, the only thing we need to do is to ask them nicely. I worded and sent a general "to do" email for all participants, and adjusted the signup form to display it after filling it out. If I think back to the last week, probably at least half of it was burned on recruitment organisation. Something which can never be underestimated on any project...

Interviews Interviews are now scheduled with 10 candidates for the next 8 days, with all the invitations sent out, from which 5 confirmed attendance. I carefully selected 10 completely different students, in terms of age, time spent at the university, studied subject and gender. I had my 1st participant in for an interview today and had a great time with lots of insights. Audio recording all interviews and later making detailed notes of the audio.

Personal and book diary preparation Both personal and book diaries are now ready and will be printed tomorrow. I working on distributing them and making them available online at the moment, with the aim that they should be available to pick up from tomorrow.

Focus group I have sent out invitations for 8 participants, carefully selected again for difference, from which I have 1 confirmed participant now. The preliminary 

Observation I started library observations in parallel at the English Faculty Library. It is a nice environment with nice people and lots of things to pick up on, so making heaps of notes. I am only doing this for 2 days, so I am pretty passive and static at the moment, but would like to engage more and generally be more mobile from next week, where all the parallel things ease up a bit.

Online survey and Hangouts Apart from thinking about them a bit, not much progress has been made. Hangouts will be scheduled soon, but the online survey is planned at the very end of the study period, so I don't even think about it for now.

This week I am interviewing mostly, so from the next post I would like to move away from planning and project preparation and focus more on reading related things in my posts. Oh and the weekly whacky bag is filling up too!

[camreading]

Recruiting for the project has now started with full steam. If you are currently a Cambridge University student, please consider helping us out and sign up. There are many ways you can help, and there are cool prizes to be won, like Amazon Kindles. More info and signup form behind this link!

EDIT: We had a very pleasant surprise in terms of participation! Nearly 200 280 students signed up the first 3 days, and the variety of the participants is great, so we have a diverse crowd which is excellent. Signup will be open for 1 more day, then we will start inviting students for interviews focus groups and hangouts, and we will start distributing the diaries soon.

[camreading]

The Weekly Whacky #1

As I said previously I will post "somewhat related" ideas, or thinking with no other reason just to dump and record and maybe to trigger further discussions. Warning! Half baked and highly suspicious things ahead! Also, most of these are triggered by reading things from usually way smarter people than me - and so always try to not get lost in attribution decay 

Making connections: Making connections between text and synthesis seems to be common core activity what is expected from students (from previous Arcadia project reports). But the connection is almost always flattened to a list of books/publications, without much immediate insight into how they affected a particular piece of writing. So creating something which allows connecting, then naming/adding meaning to the connections might not only give info on what is read, but it would also cluster/group texts in a meaningful way. Imagine visualising the connections between books and the resulting synthesis, with the added benefit of information about the nature of those connections.

Knowledge on demand: Maybe easier access to digital materials, and or knowledge in general transforms the way we think about knowledge as a whole. Maybe knowledge in our head is made up of exchangeable blocks which are retrieved and downloaded in the brain as and when needed. Synthesis is made around a topic from a certain constellation of these on-demand knowledge blocks, like cutting a slice of a cake which has constantly moving icing and jams in the middle?

Making your mark on digital documents: Electronic resources can not be transformed - ie can't put a visible bookmark, change shape of a page, or bend it. Even simple annotation is not straightforward in most cases. I mean this is fairly straightforward, but perhaps having it spelt out like this highlights that one of the reasons of students preference to read on paper is that they can transform/make their mark on a physical copy easily.

Competition around key books in the library: Can the problems around not having enough borrowable copies of key books be solved with some kind of self organised or automatic scheduling between the students outside of the university ecosystem? Something which slices up the book borrowing time for a shorter period? Or within a system like this identifying supply and demand on the short term? Is that happening in real life anyway? I would assume that a group of students with similar interests or essay titles would do this since it saves time, right?

Silence please! ...or not? Based on my understanding of previous Arcadia project data, students perceive the library as a connection hub, which gives opportunity to break the isolation of writing and offers the opportunity to meet other people in the field. But wait, historically you have to be quiet in the library? So maybe next to "Silence please" room there should be a room with "Chit chat please" in the library (or a hackerspace)?

Or we can roll this further: Construct people magnets based on topics of interest. Have meeting points in the library where people put out the topic they are discussing onto a big display so that others can see it from a distance. Ability to remotely listen to/record the conversations from home, or make notes and print off notes. Physically in my head these would be similar pods like the ones in the Porsche museum in Stuttgart, where they demonstra sounds, but with bigger dsiplay on the top for other people to see what people are talking about and comfy chairs to sit down.

[camreading]

Rough project plan, with the aim to visualise the main activities and give a high level understanding of what I will try to do.

[camreading]

Soft planning

So last week I spent on planning. Project planning for me not only means preparation for the upcoming activities, but also putting myself into the right mindset. In this instance that means the reading mindset, and so I started surrounding myself with books (more than I would do normally), I started paying attention to the different kinds of reading I do, and I started to become more conscious about small little things we tend to do during reading.

Things like efficiency. I was reading Abigail Sellen and Richard Harper's book The Myth of the Paperless Office and was fascinated to see their findings on the affordances of paper, and how people exploit the tactile nature of the paper for efficient reading. We start turning pages in parallel whilst reading, in order not to interrupt the main activity, without needing visual attention. Our fingers are capable of wondering across the page, finding the edge, grabbing a page and prepare it for turning, all without even looking at our fingers.  We can do this to be more efficient at the reading activity, and so we can focus in parallel to other activities like writing, annotating, or comparing which in most cases accompany a reading activity. In the digital world this is a bit harder, since most of the time we need to pay visual attention to the act of navigation, and even the most advanced e-readers on the market today are only catching up in this respect.

This is not closely related to this project, but I believe small things like this in the specific context of student reading at Cambridge will be key to understand, and design software or products which acknowledge and take things like this into account. So how do we get to things like this? To make this happen, apart from 'putting myself into the right mindset' I focused on 2 things:

Project plan: I created a visual project timeline, based on the amount of real time the project has, highlighting the ratio of planning, data gathering, analysis, design and artefact creation time. This will give others and me a vague framework in terms of timing, milestones and goals. I will post the project plan image in the next post, since Tumblr doesn't seem to allow inline uploading and embedding into a text post (why?)

Questions and research methodologies: To find out things we need good questions too. Not only good questions, but also questions asked in the appropriate format. I've been talking previously about how I would like to gain insights and validate those insights with quantitative methods to extrapolate. So this means that largely I will start with a small set of questions, which I will try to find answers to mainly using observation and interviews. Then as I extend my knowledge in the area, I will use a focus group and an online survey to validate those assumptions. Since I am the sole researcher in this project, that means that during the 10 weeks of the project I would only reach a limited number of people, places and situations. To somewhat balance this out I am planning 2 types of diaries in parallel, which would provide contextual information: student diaries and book diaries. Student diaries would be standard activity logs, capturing as much contextual data as possible around particular students reading activity. Book diaries would follow the route of books borrowed from libraries, requiring students to log things when the book is used. 

Using these methodologies I am aiming to establish a set of baseline data, which is received from multiple sources, for better accuracy. At the moment I am collecting all the questions I can think of initially, and categorising them based on what is the best method (of those mentioned so far) to ask that question. The Observation/Interview categories of those will be my initial ammunition.

If everything works out as planned, the baseline data coming from initially known questions, asked in multiple formats will give a strong base for the gathered data, and the additional, but yet unknown questions and findings validated later would give depth and richness in understanding. Something like this:

Observation of places and people: to gain insight and understand how things work

Interviews and casual chats: to get deeper understanding of specifics

Book and student diaries: to gather a larger volume of contextual information in parallel with observations

Focus group: to validate findings coming from observation and interviews 

Online survey: to validate findings coming from observation and interviews

Preparing things like this is immensely important, but next week I will need to get to hard planning, which involves sorting out recruitment, and observation places (library), so that all of this can actually happen. I am also still meeting a handful of extremely kind and knowledgeable people who help me pulling all this off (their names will be gathered together and included in the project hall of fame at the end) and I am also sitting in to Qualitative Research Methodologies lectures which gives me a pretty focused brush-up of how things work on the field.

[camreading]

A good summary of how ethnographic research practices inform design and product innovation in the commercial world. Can we find out more about our students with similar methods?