Category Archives: Digital Education

Association of Internet Researchers AoIR2016: Day 4

Posted on by

Today is the last day of the Association of Internet Researchers Conference 2016 – with a couple fewer sessions but I’ll be blogging throughout.

As usual this is a liveblog so corrections, additions, etc. are welcomed. 

PS-24: Rulemaking (Chair: Sandra Braman)

The DMCA Rulemaking and Digital Legal Vernaculars – Olivia G Conti, University of Wisconsin-Madison, United States of America

Apologies, I’ve joined this session late so you miss the first few minutes of what seems to have been an excellent presentation from Olivia. 

Property and ownership claims made of distinctly American values… Grounded in general ideals, evocations of the Bill of Rights. Or asking what Ben Franklin would say… Bringing the ideas of the DCMA as being contrary to the very foundations of the United Statements. Another them was the idea of once you buy something you should be able to edit as you like. Indeed a theme here is the idea of “tinkering and a liberatory endeavour”. And you see people claiming that it is a basic human right to make changes and tinker, to tweak your tractor (or whatever). Commentators are not trying to appeal to the nation state, they are trying to perform the state to make rights claims to enact the rights of the citizen in a digital world.

So, John Deere made a statement that tractro buyers have an “implied license” to their tractor, they don’t own it out right. And that raised controversies as well.

So, the final register rule was that the farmers won: they could repair their own tractors.

But the vernacular legal formations allow us to see the tensions that arise between citizens and the rights holders. And that also raises interesting issues of citizenship – and of citizenship of the state versus citizenship of the digital world.

The Case of the Missing Fair Use: A Multilingual History & Analysis of Twitter’s Policy Documentation – Amy Johnson, MIT, United States of America

This paper looks at the multilingual history and analysis of Twitter’s policy documentation. Or policies as uneven scalar tools of power alignment. And this comes from the idea of thinking of the Twitter as more than just the whole complete overarching platform. There is much research now on moderation, but understanding this type of policy allows you to understand some of the distributed nature of the platforms. Platforms draw lines when they decide which laws to tranform into policies, and then again when they think about which policies to translate.

If you look across at a list of Twitter policies, there is an English language version. Of this list it is only the Fair Use policy and the Twitter API limits that appear only in English. The API policy makes some sense, but the Fair Use policy does not. And Fair Use only appears really late – in 2014. It sets up in 2005, and many other policies come in in 2013… So what is going on?

So, here is the Twitter Fair Use Policy… Now, before I continue here, I want to say that this translation (and lack of) for this policy is unusual. Generally all companies – not just tech companies – translate into FIGS: French, Italian, German, Spanish languages. And Twitter does not do this. But this is in contrast to the translations of the platform itself. And I wanted to talk in particularly about translations into Japanese and Arabic. Now the Japanese translation came about through collaboration with a company that gave it opportunities to expand out into Japen. Arabic is not put in place until 2011, and around the Arab Spring. And the translation isn’t doen by Twitter itself but by another organisaton set up to do this. So you can see that there are other actors here playing into translations of platform and policies. So this iconic platforms are shaped in some unexpected ways.

So… I am not a lawyer but… Fair Use is a phenomenon that creates all sorts of internet lawyering. And typically there are four factors of fair use (Section 107 of US Copyright Act of 1976): purpose and character of use; nature of copyright work; amount and substantiality of portion used; effect of use on potential market for or value of copyright work. And this is very much an american law, from a legal-economic point of view. And the US is the only country that has Fair Use law.

Now there is a concept of “Fair Dealing” – mentioned in passing in Fair Use – which shares some characters. There are other countries with Fair Use law: Poland, Israel, South Korea… Well they point to the English language version. What about Japanese which has a rich reuse community on Twitter? It also points to the English policy.

So, policy are not equal in their policynesss. But why does this matter? Because this is where rule of law starts to break down… And we cannot assume that the same policies apply universally, that can’t be assumed.

But what about parody? Why bring this up? Well parody is tied up with the idea of Fair Use and creative transformation. Comedy is protected Fair Use category. And Twitter has a rich seam of parody. And indeed, if you Google for the fair use policy, the “People also ask” section has as the first question: “What is a parody account”.

Whilst Fair Use wasn’t there as a policy until 2014, parody unofficially had a policy in 2009, an official one in 2010, updates, another version in 2013 for the IPO. Biz Stone writes about, when at Google, lawyers saying about fake accounts “just say it is parody!” and the importance of parody. And indeed the parody policy has been translated much more widely than the Fair Use policy.

So, policies select bodies of law and align platforms to these bodies of law, in varying degree and depending on specific legitimation practices. Fair Use is strongly associated with US law, and embedding that in the translated policies aligns Twitter more to US law than they want to be. But parody has roots in free speech, and that is something that Twitter wishes to align itself with.

Visual Arts in Digital and Online Environments: Changing Copyright and Fair Use Practice among Institutions and Individuals Abstract – Patricia Aufderheide, Aram Sinnreich, American University, United States of America

Patricia: Aram and I have been working with the College Art Association and it brings together a wide range of professionals and practitioners in art across colleges in the US. They had a new code of conduct and we wanted to speak to them, a few months after that code of conduct was released, to see if that had changed practice and understanding. This is a group that use copyrighted work very widely. And indeed one-third of respondents avoid, abandon, or are delayed because of copyrighted work.

Aram: four-fifths of CAA members use copyrighted materials in their work, but only one fifth employ fair use to do that – most or always seek permission. And of those that use fair use there are some that always or usually use Fair Use. So there are real differences here. So, Fair Use are valued if you know about it and undestand it… but a quarter of this group aren’t sure if Fair Use is useful or not. Now there is that code of conduct. There is also some use of Creative Commons and open licenses.

Of those that use copyright materials… But 47% never use open licenses for their own work – there is a real reciprocity gap. Only 26% never use others openly licensed work. and only 10% never use others’ public domain work. Respondents value creative copying… 19 out of 20 CAA members think that creative appropriation can be “original”, and despite this group seeking permissions they also don’t feel that creative appropriation shouldn’t neccassarily require permission. This really points to an education gap within the community.

And 43% said that uncertainty about the law limits creativity. They think they would appropriate works more, they would public more, they would share work online… These mirror fair use usage!

Patricia: We surveyed this group twice in 2013 and in 2016. Much stays the same but there have been changes… In 2016, 2/3rd have heard about the code, and a third have shared that information – with peers, in teaching, with colleagues. Their associations with the concept of Fair Use are very positive.

Arem: The good news is that the code use does lead to change, even within 10 months of launch. This work was done to try and show how much impact a code of conduct has on understanding… And really there was a dramatic differences here. From the 2016 data, those who are not aware of the code, look a lot like those who are aware but have not used the code. But those who use the code, there is a real difference… And more are using fair use.

Patricia: There is one thing we did outside of the survey… There have been dramatic changes in the field. A number of universities have changed journal policies to be default Fair Use – Yale, Duke, etc. There has been a lot of change in the field. Several museums have internally changed how they create and use their materials. So, we have learned that education matters – behaviour changes with knowledge confidence. Peer support matters and validates new knowledge. Institutional action, well publicized, matters .The newest are most likely to change quickly, but the most veteran are in the best position – it is important to have those influencers on board… And teachers need to bring this into their teaching practice.

Panel Q&A

Q1) How many are artists versus other roles?

A1 – Patricia) About 15% are artists, and they tend to be more positive towards fair use.

Q2) I was curious about changes that took place…

A2 – Arem) We couldn’t ask whether the code made you change your practice… But we could ask whether they had used fair use before and after…

Q3) You’ve made this code for the US CAA, have you shared that more widely…

A3 – Patricia) Many of the CAA members work internationally, but the effectiveness of this code in the US context is that it is about interpreting US Fair Use law – it is not a legal document but it has been reviewed by lawyers. But copyright is territorial which makes this less useful internationally as a document. If copyright was more straightforward, that would be great. There are rights of quotation elsewhere, there is fair dealing… And Canadian law looks more like Fair Use. But the US is very litigious so if something passes Fair Use checking, that’s pretty good elsewhere… But otherwise it is all quite territorial.

A3 – Arem) You can see in data we hold that international practitioners have quite different attitudes to American CAA members.

Q4) You talked about the code, and changes in practice. When I talk to filmmakers and documentary makers in Germany they were aware of Fair Use rights but didn’t use them as they are dependent on TV companies buy them and want every part of rights cleared… They don’t want to hurt relationships.

A4 – Patricia) We always do studies before changes and it is always about reputation and relationship concerns… Fair Use only applies if you can obtain the materials independently… But then the question may be that will rights holders be pissed off next time you need to licence content. What everyone told me was that we can do this but it won’t make any difference…

Chair) I understand that, but that question is about use later on, and demonstration of rights clearance.

A4 – Patricia) This is where change in US errors and omissions insurance makes a difference – that protects them. The film and television makers code of conduct helped insurers engage and feel confident to provide that new type of insurance clause.

Q5) With US platforms, as someone in Norway, it can be hard to understand what you can and cannot access and use on, for instance, in YouTube. Also will algorithmic filtering processes of platforms take into account that they deal with content in different territories?

A5 – Arem) I have spoken to Google Council about that issue of filtering by law – there is no difference there… But monitoring

A5 – Amy) I have written about legal fictions before… They are useful for thinking about what a “reasonable person” – and that can be vulnerable by jury and location so writing that into policies helps to shape that.

A5 – Patricia) The jurisdiction is where you create, not where the work is from…

Q6) There is an indecency case in France which they want to try in French court, but Facebook wants it tried in US court. What might the impact on copyright be?

A6 – Arem) A great question but this type of jurisdictional law has been discussed for over 10 years without any clear conclusion.

A6 – Patricia) This is a European issue too – Germany has good exceptions and limitations, France has horrible exceptions and limitations. There is a real challenge for pan European law.

Q7) Did you look at all of impact on advocacy groups who encouraged writing in/completion of replies on DCMA. And was there any big difference between the farmers and car owners?

A7) There was a lot of discussion on the digital right to repair site, and that probably did have an impact. I did work on Net Neutrality before. But in any of those cases I take out boiler plate, and see what they add directly – but there is a whole other paper to be done on boiler plate texts and how they shape responses and terms of additional comments. It wasn’t that easy to distinguish between farmers and car owners, but it was interesting how individuals established credibility. For farmers they talked abot the value of fixing their own equipment, of being independent, of history of ownership. Car mechanics, by contrast, establish technical expertise.

Q8) As a follow up: farmers will have had a long debate over genetically modified seeds – and the right to tinker in different ways…

A8) I didn’t see that reflected in the comments, but there may well be a bigger issue around micromanagement of practices.

Q9) Olivia, I was wondering if you were considering not only the rhetorical arguements of users, what about the way the techniques and tactics they used are received on the other side… What are the effective tactics there, or locate the limits of the effectiveness of the layperson vernacular stategies?

A9) My goal was to see what frames of arguements looked most effective. I think in the case of the John Deere DCMA case that wasn’t that conclusive. It can be really hard to separate the NGO from the individual – especially when NGOs submit huge collections of individual responses. I did a case study on non-consensual pornography was more conclusive in terms of strategies that was effective. The discourses I look at don’t look like legal discourse but I look at the tone and content people use. So, on revenge porn, the law doesn’t really reflect user practice for instance.

Q10) For Amy, I was wondering… Is the problem that Fair Use isn’t translated… Or the law behind that?

A10 – Amy) I think Twitter in particular have found themselves in a weird middle space… Then the exceptions wouldn’t come up. But having it in English is the odd piece. That policy seems to speak specifically to Americans… But you could argue they are trying to impose (maybe that’s a bit too strong) on all English speaking territory. On YouTube all of the policies are translated into the same languages, including Fair Use.

Q11) I’m fascinated in vernacular understanding and then the experts who are in the round tables, who specialise in these areas. How do you see vernacular discourse use in more closed/smaller settings?

A11 – Olivia) I haven’t been able to take this up as so many of those spaces are opaque. But in the 2012 rule making there were some direct quotes from remixers. And there a suggestion around DVD use that people should videotape the TV screen… and that seemed unreasonably onorous…

Chair) Do you forsee a next stage where you get to be in those rooms and do more on that?

A11 – Olivia) I’d love to do some ethnographic studies, to get more involved.

A11 – Patricia) I was in Washington for the DMCA hearings and those are some of the most fun things I go to. I know that the documentary filmmakers have complained about cost of participating… But a technician from the industry gave 30 minutes of evidence on the 40 technical steps to handle analogue film pieces of information… And to show that it’s not actually broadcast quality. It made them gasp. It was devastating and very visual information, and they cited it in their ruling… And similarly in John Deere case the car technicians made impact. By contrast a teacher came in to explain why copying material was important for teaching, but she didn’t have either people or evidence of what the difference is in the classroom.

Q12) I have an interesting case if anyone wants to look at it, around Wikipedia’s Fair Use issues around multimedia. Volunteers take pre-emptively being stricter as they don’t want lawyers to come in on that… And the Wikipedia policies there. There is also automation through bots to delete content without clear Fair Use exception.

A12 – Arem) I’ve seen Fair Use misappropriated on Wikipedia… Copyright images used at low resolution and claimed as Fair Use…

A12- Patricia) Wikimania has all these people who don’t want to deal with law on copyright at all! Wikimedia lawyers are in an a really difficult position.

Share/Bookmark

Association of Internet Researchers AoIR 2016: Day Two

Posted on by

Today I am again at the Association of Internet Researchers AoIR 2016 Conference in Berlin. Yesterday we had workshops, today the conference kicks off properly. Follow the tweets at: #aoir2016.

As usual this is a liveblog so all comments and corrections are very much welcomed. 

Platform Studies: The Rules of Engagement (Chair: Jean Burgess, QUT)

How affordances arise through relations between platforms, their different types of users, and what they do to the technology – Taina Bucher (University of Copenhagen) and Anne Helmond (University of Amsterdam)

Taina: Hearts on Twitter: In 2015 Twitter moved from stars to hearts, changing the affordances of the platform. They stated that they wanted to make the platform more accessible to new users, but that impacted on existing users.

Today we are going to talk about conceptualising affordances. In it’s original meaning an affordance is conceived of as a relational property (Gibson). For Norman perceived affordances were more the concern – thinking about how objects can exhibit or constrain particular actions. Affordances are not just the visual clues or possibilities, but can be felt. Gaver talks about these technology affordances. There are also social affordances – talked about my many – mainly about how poor technological affordances have impact on societies. It is mainly about impact of technology and how it can contain and constrain sociality. And finally we have communicative affordances (Hutchby), how technological affordances impact on communities and communications of practices.

So, what about platform changes? If we think about design affordances, we can see that there are different ways to understand this. The official reason for the design was given as about the audience, affording sociality of community and practices.

Affordances continues to play an important role in media and social media research. They tend to be conceptualised as either high-level or low-level affordances, with ontological and epistemological differences:

  • High: affordance in the relation – actions enabled or constrained
  • Low: affordance in the technical features of the user interface – reference to Gibson but they vary in where and when affordances are seen, and what features are supposed to enable or constrain.

Anne: We want to now turn to platform-sensitive approach, expanding the notion of the user –> different types of platform users, end-users, developers, researchers and advertisers – there is a real diversity of users and user needs and experiences here (see Gillespie on platforms. So, in the case of Twitter there are many users and many agendas – and multiple interfaces. Platforms are dynamic environments – and that differentiates social media platforms from Gibson’s environmental platforms. Computational systems driving media platforms are different, social media platforms adjust interfaces to their users through personalisation, A/B testing, algorithmically organised (e.g. Twitter recommending people to follow based on interests and actions).

In order to take a relational view of affordances, and do that justice, we also need to understand what users afford to the platforms – as they contribute, create content, provide data that enables to use and development and income (through advertisers) for the platform. Returning to Twitter… The platform affords different things for different people

Taking medium-specificity of platforms into account we can revisit earlier conceptions of affordance and critically analyse how they may be employed or translated to platform environments. Platform users are diverse and multiple, and relationships are multidirectional, with users contributing back to the platform. And those different users have different agendas around affordances – and in our Twitter case study, for instance, that includes developers and advertisers, users who are interested in affordances to measure user engagement.

How the social media APIs that scholars so often use for research are—for commercial reasons—skewed positively toward ‘connection’ and thus make it difficult to understand practices of ‘disconnection’ – Nicolas John (Hebrew University of Israel) and Asaf Nissenbaum (Hebrew University of Israel)

Consider this… On Facebook…If you add someone as a friend they are notified. If you unfriend them, they do not. If you post something you see it in your feed, if you delete it it is not broadcast. They have a page called World of Friends – they don’t have one called World of Enemies. And Facebook does not take kindly to app creators who seek to surface unfriending and removal of content. And Facebook is, like other social media platforms, therefore significantly biased towards positive friending and sharing actions. And that has implications for norms and for our research in these spaces.

One of our key questions here is what can’t we know about

Agnotology is defined as the study of ignorance. Robert Proctor talks about this in three terms: native state – childhood for instance; strategic ploy – e.g. the tobacco industry on health for years; lost realm – the knowledge that we cease to hold, that we loose.

I won’t go into detail on critiques of APIs for social science research, but as an overview the main critiques are:

  1. APIs are restrictive – they can cost money, we are limited to a percentage of the whole – Burgess and Bruns 2015; Bucher 2013; Bruns 2013; Driscoll and Walker
  2. APIs are opaque
  3. APIs can change with little notice (and do)
  4. Omitted data – Baym 2013 – now our point is that these platforms collect this data but do not share it.
  5. Bias to present – boyd and Crawford 2012

Asaf: Our methodology was to look at some of the most popular social media spaces and their APIs. We were were looking at connectivity in these spaces – liking, sharing, etc. And we also looked for the opposite traits – unliking, deletion, etc. We found that social media had very little data, if any, on “negative” traits – and we’ll look at this across three areas: other people and their content; me and my content; commercial users and their crowds.

Other people and their content – APIs tend to supply basic connectivity – friends/following, grouping, likes. Almost no historical content – except Facebook which shares when a user has liked a page. Current state only – disconnections are not accounted for. There is a reason to not know this data – privacy concerns perhaps – but that doesn’t explain my not being able to find this sort of information about my own profile.

Me and my content – negative traits and actions are hidden even from ourselves. Success is measured – likes and sharin, of you or by you. Decline is not – disconnections are lost connections… except on Twitter where you can see analytics of followers – but no names there, and not in the API. So we are losing who we once were but are not anymore. Social network sites do not see fit to share information over time… Lacking disconnection data is an idealogical and commercial issue.

Commercial users and their crowds – these users can see much more of their histories, and the negative actions online. They have a different regime of access in many cases, with the ups and downs revealed – though you may need to pay for access. Negative feedback receives special attention. Facebook offers the most detailed information on usage – including blocking and unliking information. Customers know more than users, or Pages vs. Groups.

Nicholas: So, implications. From what Asaf has shared shows the risk for API-based research… Where researchers’ work may be shaped by the affordances of the API being used. Any attempt to capture negative actions – unlikes, choices to leave or unfriend. If we can’t use APIs to measure social media phenomena, we have to use other means. So, unfriending is understood through surveys – time consuming and problematic. And that can put you off exploring these spaces – it limits research. The advertiser-friends user experience distorts the space – it’s like the stock market only reporting the rises except for a few super wealthy users who get the full picture.

A biography of Twitter (a story told through the intertwined stories of its key features and the social norms that give them meaning, drawing on archival material and oral history interviews with users) – Jean Burgess (Queensland University of Technology) and Nancy Baym (Microsoft Research)

I want to start by talking about what I mean by platforms, and what I mean by biographies. Here platforms are these social media platforms that afford particular possibilities, they enable and shape society – we heard about the platformisation of society last night – but their governance, affordances, are shaped by their own economic existance. They are shaping and mediating socio-cultural experience and we need to better to understand the values and socio-cultural concerns of the platforms. By platform studies we mean treating social media platforms as spaces to study in their own rights: as institutions, as mediating forces in the environment.

So, why “biography” here? First we argue that whilst biographical forms tend to be reserved for individuals (occasionally companies and race horses), they are about putting the subject in context of relationships, place in time, and that the context shapes the subject. Biographies are always partial though – based on unreliable interviews and information, they quickly go out of date, and just as we cannot get inside the heads of those who are subjects of biographies, we cannot get inside many of the companies at the heart of social media platforms. But (after Richard Rogers) understanding changes helps us to understand the platform.

So, in our forthcoming book, Twitter: A Biography (NYU 2017), we will look at competing and converging desires around e.g the @, RT, #. Twitter’s key feature set are key characters in it’s biography. Each has been a rich site of competing cultures and norms. We drew extensively on the Internet Archives, bloggers, and interviews with a range of users of the platform.

Nancy: When we interviewed people we downloaded their archive with them and talked through their behaviour and how it had changed – and many of those features and changes emerged from that. What came out strongly is that noone knows what Twitter is for – not just amongst users but also amongst the creators – you see that today with Jack Dorsey and Anne Richards. The heart of this issue is about whether Twitter is about sociality and fun, or is it a very important site for sharing important news and events. Users try to negotiate why they need this space, what is it for… They start squabling saying “Twitter, you are doing it wrong!”… Changes come with backlash and response, changed decisions from Twitter… But that is also accompanied by the media coverage of Twitter, but also the third party platforms build on Twitter.

So the “@” is at the heart of Twitter for sociality and Twitter for information distribution. It was imported from other spaces – IRC most obviously – as with other features. One of the earliest things Twitter incorporated was the @ and the links back.. You have things like originally you could see everyone’s @ replies and that led to feed clutter – although some liked seeing unexpected messages like this. So, Twitter made a change so you could choose. And then they changed again to automatically not see replies from those you don’t follow. So people worked around that with “.@” – which created conflict between the needs of the users, the ways they make it usable, and the way the platform wants to make the space less confusing to new users.

The “RT” gave credit to people for their words, and preserved integrity of words. At first this wasn’t there and so you had huge variance – the RT, the manually spelled out retweet, the hat tip (HT). Technical changes were made, then you saw the number of retweets emerging as a measure of success and changing cultures and practices.

The “#” is hugely disputed – it emerged through hashtag.org: you couldn’t follow them in Twitter at first but they incorporated it to fend off third party tools. They are beloved by techies, and hated by user experience designers. And they are useful but they are also easily coopted by trolls – as we’ve seen on our own hashtag.

Insights into the actual uses to which audience data analytics are put by content creators in the new screen ecology (and the limitations of these analytics) – Stuart Cunningham (QUT) and David Craig (USC Annenberg School for Communication and Journalism)

The algorithmic culture is well understood as a part of our culture. There are around 150 items on Tarleton Gillespie and Nick Seaver’s recent reading list and the literature is growing rapidly. We want to bring back a bounded sense of agency in the context of online creatives.

What do I mean by “online creatives”? Well we are looking at social media entertainment – a “new screen ecology” (Cunningham and Silver 2013; 2015) shaped by new online creatives who are professionalising and monetising on platforms like YouTube, as opposed to professional spaces, e.g. Netflix. YouTube has more than 1 billion users, with revenue in 2015 estimated at $4 billion per year. And there are a large number of online creatives earning significant incomes from their content in these spaces.

Previously online creatives were bound up with ideas of democratic participative cultures but we want to offer an immanent critique of the limits of data analytics/algorithmic culture in shaping SME from with the industry on both the creator (bottom up) and platform (top down) side. This is an approach to social criticism exposes the way reality conflicts not with some “transcendent” concept of rationality but with its own avowed norms, drawing on Foucault’s work on power and domination.

We undertook a large number of interviews and from that I’m going to throw some quotes at you… There is talk of information overload – of what one might do as an online creative presented with a wealth of data. Creatives talk about the “non-scalable practices” – the importance and time required to engage with fans and subscribers. Creatives talk about at least half of a working week being spent on high touch work like responding to comments, managing trolls, and dealing with challenging responses (especially with creators whose kids are engaged in their content).

We also see cross-platform engagement – and an associated major scaling in workload. There is a volume issue on Facebook, and the use of Twitter to manage that. There is also a sense of unintended consequences – scale has destroyed value. Income might be $1 or $2 for 100,000s or millions of views. There are inherent limits to algorithmic culture… But people enjoy being part of it and reflect a real entrepreneurial culture.

In one or tow sentences, the history of YouTube can be seen as a sort of clash of NoCal and SoCal cultures. Again, no-one knows what it is for. And that conflict has been there for ten years. And you also have the MCNs (Multi-Contact Networks) who are caught like the meat in the sandwich here.

Panel Q&A

Q1) I was wondering about user needs and how that factors in. You all drew upon it to an extent… And the dissatisfaction of users around whether needs are listened to or not was evident in some of the case studies here. I wanted to ask about that.

A1 – Nancy) There are lots of users, and users have different needs. When platforms change and users are angry, others are happy. We have different users with very different needs… Both of those perspectives are user needs, they both call for responses to make their needs possible… The conflict and challenges, how platforms respond to those tensions and how efforts to respond raise new tensions… that’s really at the heart here.

A1 – Jean) In our historical work we’ve also seen that some users voices can really overpower others – there are influential users and they sometimes drown out other voices, and I don’t want to stereotype here but often technical voices drown out those more concerned with relationships and intimacy.

Q2) You talked about platforms and how they developed (and I’m afraid I didn’t catch the rest of this question…)

A2 – David) There are multilateral conflicts about what features to include and exclude… And what is interesting is thinking about what ideas fail… With creators you see economic dependence on platforms and affordances – e.g. versus PGC (Professionally Generated Content).

A2 – Nicholas) I don’t know what user needs are in a broader sense, but everyone wants to know who unfriended them, who deleted them… And a dislike button, or an unlike button… The response was strong but “this post makes me sad” doesn’t answer that and there is no “you bastard for posting that!” button.

Q3) Would it be beneficial to expose unfriending/negative traits?

A3 – Nicholas) I can think of a use case for why unfriending would be useful – for instance wouldn’t it be useful to understand unfriending around the US elections. That data is captured – Facebook know – but we cannot access it to research it.

A3 – Stuart) It might be good for researchers, but is it in the public good? In Europe and with the Right to be Forgotten should we limit further the data availability…

A3 – Nancy) I think the challenge is that mismatch of only sharing good things, not sharing and allowing exploration of negative contact and activity.

A3 – Jean) There are business reasons for positivity versus negativity, but it is also about how the platforms imagine their customers and audiences.

Q4) I was intrigued by the idea of the “Medium specificity of platforms” – what would that be? I’ve been thinking about devices and interfaces and how they are accessed… We have what we think of as a range but actually we are used to using really one or two platforms – e.g. Apple iPhone – in terms of design, icons, etc. and the possibilities of interface is, and what happens when something is made impossible by the interface.

A4 – Anne) When the “medium specificity” we are talking about the platform itself as medium. Moving beyond end user and user experience. We wanted to take into account the role of the user – the platform also has interfaces for developers, for advertisers, etc. and we wanted to think about those multiple interfaces, where they connect, how they connect, etc.

A4 – Taina) It’s a great point about medium specitivity but for me it’s more about platform specifity.

A4 – Jean) The integration of mobile web means the phone iOS has a major role here…

A4 – Nancy) We did some work with couples who brought in their phones, and when one had an Apple and one had an Android phone we actually found that they often weren’t aware of what was possible in the social media apps as the interfaces are so different between the different mobile operating systems and interfaces.

Q5) Can you talk about algorithmic content and content innovation?

A5 – David) In our work with YouTube we see forms of innovation that are very platform specific around things like Vine and Instagram. And we also see counter-industrial forms and practices. So, in the US, we see blogging and first person accounts of lives… beauty, unboxing, etc. But if you map content innovation you see (similarly) this taking the form of gaps in mainstream culture – in India that’s stand up comedy for instance. Algorithms are then looking for qualities and connections based on what else is being accessed – creating a virtual circle…

Q6) Can we think of platforms as instable, about platforms having not quite such a uniform sense of purpose and direction…

A6 – Stuart) Most platforms are very big in terms of their finance… If you compare that to 20 years ago the big companies knew what they were doing! Things are much more volatile…

A6 – Jean) That’s very common in the sector, except maybe on Facebook… Maybe.

Share/Bookmark

Association of Internet Researchers AoIR 2016 РDay 1 РJos̩ van Dijck Keynote

Posted on by

If you’ve been following my blog today you will know that I’m in Berlin for the Association of Internet Researchers AoIR 2016 (#aoir2016) Conference, at Humboldt University. As this first day has mainly been about workshops – and I’ve been in a full day long Digital Methods workshop – we do have our first conference keynote this evening. And as it looks a bit different to my workshop blog, I thought a new post was in order.

As usual, this is a live blog post so corrections, comments, etc. are all welcomed. This session is also being videoed so you will probably want to refer to that once it becomes available as the authoritative record of the session. 

Keynote: The Platform Society – José van Dijck (University of Amsterdam) with Session Chair: Jennifer Stromer-Galley

 

Share/Bookmark

Association of Internet Researchers AoIR 2016: Day 1 – Workshops

Posted on by

After a few weeks of leave I’m now back and spending most of this week at the Association of Internet Researchers (AoIR) Conference 2016. I’m hugely excited to be here as the programme looks excellent with a really wide range of internet research being presented and discussed. I’ll be liveblogging throughout the week starting with today’s workshops.

I am booked into the Digital Methods in Internet Research: A Sampling Menu workshop, although I may be switching session at lunchtime to attend the Internet rules… for Higher Education workshop this afternoon.

The Digital Methods workshop is being chaired by Patrik Wikstrom (Digital Media Research Centre, Queensland University of Technology, Australia) and the speakers are:

  • Erik Borra (Digital Methods Initiative, University of Amsterdam, the Netherlands),
  • Axel Bruns (Digital Media Research Centre, Queensland University of Technology, Australia),
  • Jean Burgess (Digital Media Research Centre, Queensland University of Technology, Australia),
  • Carolin Gerlitz (University of Siegen, Germany),
  • Anne Helmond (Digital Methods Initiative, University of Amsterdam, the Netherlands),
  • Ariadna Matamoros Fernandez (Digital Media Research Centre, Queensland University of Technology, Australia),
  • Peta Mitchell (Digital Media Research Centre, Queensland University of Technology, Australia),
  • Richard Rogers (Digital Methods Initiative, University of Amsterdam, the Netherlands),
  • Fernando N. van der Vlist (Digital Methods Initiative, University of Amsterdam, the Netherlands),
  • Esther Weltevrede (Digital Methods Initiative, University of Amsterdam, the Netherlands).

I’ll be taking notes throughout but the session materials are also available here: http://tinyurl.com/aoir2016-digmethods/.

Patrik: We are in for a long and exciting day! I won’t introduce all the speakers as we won’t have time!

Conceptual Introduction: Situating Digital Methods (Richard Rogers)

My name is Richard Rogers, I’m professor of new media and digital culture at the University of Amsterdam and I have the pleasure of introducing today’s session. So I’m going to do two things, I’ll be situating digital methods in internet-related research, and then taking you through some digital methods.

I would like to situate digital methods as a third era of internet research… I think all of these eras thrive and overlap but they are differentiated.

  1. Web of Cyberspace (1994-2000): Cyberstudies was an effort to see difference in the internet, the virtual as distinct from the real. I’d situate this largely in the 90’s and the work of Steve Jones and Steve (?).
  2. Web as Virtual Society? (2000-2007) saw virtual as part of the real. Offline as baseline and “virtual methods” with work around the digital economy, the digital divide…
  3. Web as societal data (2007-) is about “virtual as indication of the real. Online as baseline.

Right now we use online data about society and culture to make “grounded” claims.

So, if we look at Allrecipes.com Thanksgiving recipe searches on a map we get some idea of regional preference, or we look at Google data in more depth, we get this idea of internet data as grounding for understanding culture, society, tastes.

So, we had this turn in around 2008 to “web as data” as a concept. When this idea was first introduced not all were comfortable with the concept. Mike Thelwell et al (2005) talked about the importance of grounding the data from the internet. So, for instance, Google’s flu trends can be compared to Wikipedia traffic etc. And with these trends we also get the idea of “the internet knows first”, with the web predicting other sources of data.

Now I do want to talk about digital methods in the context of digital humanities data and methods. Lev Manovich talks about Cultural Analytics. It is concerned with digitised cultural materials with materials clusterable in a sort of art historical way – by hue, style, etc. And so this is a sort of big data approach that substitutes “continuous change” for periodisation and categorisation for continuation. So, this approach can, for instance, be applied to Instagram (Selfiexploration), looking at mood, aesthetics, etc. And then we have Culturenomics, mainly through the Google Ngram Viewer. A lot of linguists use this to understand subtle differences as part of distance reading of large corpuses.

And I also want to talk about e-social sciences data and method. Here we have Webometrics (Thelwell et al) with links as reputational markers. The other tradition here is Altmetrics (Priem et al), which uses online data to do citation analysis, with social media data.

So, at least initially, the idea behind digital methods was to be in a different space. The study of online digital objects, and also natively online method – methods developed for the medium. And natively digital is meant in a computing sense here. In computing software has a native mode when it is written for a specific processor, so these are methods specifically created for the digital medium. We also have digitized methods, those which have been imported and migrated methods adapted slightly to the online.

Generally speaking there is a sort of protocol for digital methods: Which objects and data are available? (links, tags, timestamps); how do dominant devices handle them? etc.

I will talk about some methods here:

1. Hyperlink

For the hyperlink analysis there are several methods. The Issue Crawler software, still running and working, enable you to see links between pages, direction of linking, aspirational linking… For example a visualisation of an Armenian NGO shows the dynamics of an issue network showing politics of association.

The other method that can be used here takes a list of sensitive sites, using Issue Crawler, then parse it through an internet censorship service. And variations on this that indicate how successful attempts at internet censorship are. We do work on Iran and China and I should say that we are always quite thoughtful about how we publish these results because of their sensitivity.

2. The website as archived object

We have the Internet Archive and we have individual archived web sites. Both are useful but researcher use is not terribly signficant so we have been doing work on this. See also a YouTube video called “Google and the politics of tabs” – a technique to create a movie of the evolution of a webpage in the style of timelapse photography. I will be publishing soon about this technique.

But we have also been looking at historical hyperlink analysis – giving you that context that you won’t see represented in archives directly. This shows the connections between sites at a previous point in time. We also discovered that the “Ghostery” plugin can also be used with archived websites – for trackers and for code. So you can see the evolution and use of trackers on any website/set of websites.

6. Wikipedia as cultural reference

Note: the numbering is from a headline list of 10, hence the odd numbering… 

We have been looking at the evolution of Wikipedia pages, understanding how they change. It seems that pages shift from neutral to national points of view… So we looked at Srebenica and how that is represented. The pages here have different names, indicating difference in the politics of memory and reconciliation. We have developed a triangulation tool that grabs links and references and compares them across different pages. We also developed comparative image analysis that lets you see which images are shared across articles.

7. Facebook and other social networking sites

Facebook is, as you probably well know, is a social media platform that is relatively difficult to pin down at a moment in time. Trying to pin down the history of Facebook find that very hard – it hasn’t been in the Internet Archive for four years, the site changes all the time. We have developed two approaches: one for social media profiles and interest data as means of stufying cultural taste ad political preference or “Postdemographics”; And “Networked content analysis” which uses social media activity data as means of studying “most engaged with content” – that helps with the fact that profiles are no longer available via the API. To some extend the API drives the research, but then taking a digital methods approach we need to work with the medium, find which possibilities are there for research.

So, one of the projects undertaken with in this space was elFriendo, a MySpace-based project which looked at the cultural tastes of “friends” of Obama and McCain during their presidential race. For instance Obama’s friends best liked Lost and The Daily Show on TV, McCain’s liked Desperate Housewives, America’s Next Top Model, etc. Very different cultures and interests.

Now the Networked Content Analysis approach, where you quantify and then analyse, works well with Facebook. You can look at pages and use data from the API to understand the pages and groups that liked each other, to compare memberships of groups etc. (at the time you were able to do this). In this process you could see specific administrator names, and we did this with right wing data working with a group called Hope not Hate, who recognised many of the names that emerged here. Looking at most liked content from groups you also see the shared values, cultural issues, etc.

So, you could see two areas of Facebook Studies, Facebook I (2006-2011) about presentation of self: profiles and interests studies (with ethics); Facebook II (2011-) which is more about social movements. I think many social media platforms are following this shift – or would like to. So in Instagram Studies the Instagram I (2010-2014) was about selfie culture, but has shifed to Instagram II (2014-) concerned with antagonistic hashtag use for instance.

Twitter has done this and gone further… Twitter I (2006-2009) was about urban lifestyle tool (origins) and “banal” lunch tweets – their own tagline of “what are you doing?”, a connectivist space; Twitter II (2009-2012) has moved to elections, disasters and revolutions. The tagline is “what’s happening?” and we have metrics “trending topics”; Twitter III (2012-) sees this as a generic resource tool with commodification of data, stock market predictions, elections, etc.

So, I want to finish by talking about work on Twitter as a storytelling machine for remote event analysis. This is an approach we developed some years ago around the Iran event crisis. We made a tweet collection around a single Twitter hashtag – which is no longer done – and then ordered by most retweeted (top 3 for each day) and presented in chronological (not reverse) order. And we then showed those in huge displays around the world…

To take you back to June 2009… Mousavi holds an emergency press conference. Voter turn out is 80%. SMS is down. Mousavi’s website and Facebook are blocked. Police use pepper spray… The first 20 days of most popular tweets is a good succinct summary of the events.

So, I’ve taken you on a whistle stop tour of methods. I don’t know if we are coming to the end of this. I was having a conversation the other day that the Web 2.0 days are over really, the idea that the web is readily accessible, that APIs and data is there to be scraped… That’s really changing. This is one of the reasons the app space is so hard to research. We are moving again to user studies to an extent. What the Chinese researchers are doing involves convoluted processes to getting the data for instance. But there are so many areas of research that can still be done. Issue Crawler is still out there and other tools are available at tools.digitalmethods.net.

Twitter studies with DMI-TCAT (Erik Borra)

I’m going to be talking about how we can use the DMI-TCAT tool to do Twitter Studies. I am here with Emile den Tex, one of the original developers of this tool, alongside Eric Borra.

So, what is DMI-TCAT? It is the Digital Methods Initiative Twitter Capture and Analysis Toolset, a server side tool which tries to capture robust and reproducible data capture and analysis. The design is based on two ideas: that captured datasets can be refined in different ways; and that the datasets can be analysed in different ways. Although we developed this tool, it is also in use elsewhere, particularly in the US and Australia.

So, how do we actually capture Twitter data? Some of you will have some experience of trying to do this. As researchers we don’t just want the data, we also want to look at the platform in itself. If you are in industry you get Twitter data through a “data partner”, the biggest of which by far is GNIP – owned by Twitter as of the last two years – then you just pay for it. But it is pricey. If you are a researcher you can go to an academic data partner – DiscoverText or Hexagon – and they are also resellers but they are less costly. And then the third route is the publicly available data – REST APIs, Search API, Streaming APIs. These are, to an extent, the authentic user perspective as most people use these… We have built around these but the available data and APIs shape and constrain the design and the data.

For instance the “Search API” prioritises “relevance” over “completeness” – but as academics we don’t know how “relevance” is being defined here. If you want to do representative research then completeness may be most important. If you want to look at how Twitter prioritises the data, then that Search API may be most relevant. You also have to understand rate limits… This can constrain research, as different data has different rate limits.

So there are many layers of technical mediation here, across three big actors: Twitter platform – and the APIs and technical data interfaces; DMI-TCAT (extraction); Output types. And those APIs and technical data interfaces are significant mediators here, and important to understand their implications in our work as researchers.

So, onto the DMI-TCAT tool itself – more on this in Borra & Reider (2014) (doi:10.1108/AJIM-09-2013-0094). They talk about “programmed method” and the idea of the methodological implications of the technical architecture.

What can one learn if one looks at Twitter through this “programmed method”? Well (1) Twitter users can change their Twitter handle, but their ids will remain identical – sounds basic but its important to understand when collecting data. (2) the length of a Tweet may vary beyond maximum of 140 characters (mentions and urls); (3) native retweets may have their top level text property stortened. (4) Unexpected limitations  support for new emoji characters can be problematic. (5) It is possible to retrieve a deleted tweet.

So, for example, a tweet can vary beyond 140 characters. The Retweet of an original post may be abbreviated… Now we don’t want that, we want it to look as it would to a user. So, we capture it in our tool in the non-truncated version.

And, on the issue of deletion and witholding. There are tweets deleted by users, and their are tweets which are withheld by the platform – and the withholding is a country by country issue. But you can see tweets only available in some countries. A project that uses this information is “Politwoops” (http://politwoops.sunlightfoundation.com/) which captures tweets deleted by US politicians, that lets you filter to specific states, party, position. Now there is an ethical discussion to be had here… We don’t know why tweets are deleted… We could at least talk about it.

So, the tool captures Twitter data in two ways. Firstly there is the direct capture capabilities (via web front-end) which allows tracking of users and capture of public tweets posted by these users; tracking particular terms or keywords, including hashtags; get a small random (approx 1%) of all public statuses. Secondary capture capabilities (via scripts) allows further exploration, including user ids, deleted tweets etc.

Twitter as a platform has a very formalised idea of sociality, the types of connections, parameters, etc. When we use the term “user” we mean it in the platform defined object meaning of the word.

Secondary analytical capabilities, via script, also allows further work:

  1. support for geographical polygons to delineate geographical regions for tracking particular terms or keywords, including hashtags.
  2. Built-in URL expander, following shortened URLs to their destination. Allowing further analysis, including of which statuses are pointing to the same URLs.
  3. Download media (e.g. videos and images (attached to particular Tweets).

So, we have this tool but what sort of studies might we do with Twitter? Some ideas to get you thinking:

  1. Hashtag analysis – users, devices etc. Why? They are often embedded in social issues.
  2. Mentions analysis – users mentioned in contexts, associations, etc. allowing you to e.g. identify expertise.
  3. Retweet analysis – most retweeted per day.
  4. URL analysis – the content that is most referenced.

So Emile will now go through the tool and how you’d use it in this way…

Emile: I’m going to walk through some main features of the DMI TCAT tool. We are going to use a demo site (http://tcatdemo.emiledentex.nl/analysis/) and look at some Trump tweets…

Note: I won’t blog everything here as it is a walkthrough, but we are playing with timestamps (the tool uses UTC), search terms etc. We are exploring hashtag frequency… In that list you can see Bengazi, tpp, etc. Now, once you see a common hashtag, you can go back and query the dataset again for that hashtag/search terms… And you can filter down… And look at “identical tweets” to found the most retweeted content. 

Emile: Eric called this a list making tool – it sounds dull but it is so useful… And you can then put the data through other tools. You can put tweets into Gephi. Or you can do exploration… We looked at Getty Parks project, scraped images, reverse Google image searched those images to find the originals, checked the metadata for the camera used, and investigated whether the cost of a camera was related to the success in distributing an image…

Richard: It was a critique of user generated content.

Analysing Social Media Data with TCAT and Tableau (Axel Bruns)

Analysing Network Dynamics with Agent Based Models (Patrik Wikström)

Tracking the Trackers (Anne Helmond, Carolin Gerlitz, Esther Weltevrede and Fernando van der Vlist)

Multiplatform Issue Mapping (Jean Burgess & Ariadna Matamoros Fernandez)

Analysing and visualising geospatial data (Peta Mitchell)

 

Share/Bookmark

eLearning@ed/LTW Monthly Meet Up #4: Learning Design

Posted on by

This is a very belated posting of my liveblog notes from the eLearning@Ed/LTW Monthly Meet Up #4 on Learning Design which took place on 25th April 2016. You can find further information on the event, and all of our speakers’ slides, on the eLearning@ed wiki.

Despite the delay in posting these notes, the usual cautionary notes apply, and that all corrections, additions, etc. are very much welcomed. 

Becoming an ELDeR – Fiona Hale, Senior eLearning Advisor, IS

Unfortunately I missed capturing notes for the very beginning of Fiona’s talk but I did catch most of it. As context please be aware that she was talking about a significant and important piece of work on Learning Design, including a scoping report by Fiona, which has been taking place over the last year. My notes start as she addresses the preferred formats for learning design training… 

We found that two-day workshops provided space to think, to collaborate, and had the opportunity to both gain new knowledge and apply it on the same day. And also really useful for academic staff to understand the range of colleagues in the room, knowing who they could and should follow up with.

Scoping report recommended developing reusable and collaborative learning design as a new university services within IS, which positions the learning design framework as a scaffold, support staff as facilitators, etc.

There are many recommendations here but in particular I wanted to talk about the importance of workshops being team based and collaborative in approach – bringing together programme team, course team, admin, LT, peer, student, IAD, IS Support librarian, IS EDE, Facilitator, all in the room. Also part of staff development, reward and recognition – tying into UKSPF (HEA) and the Edinburgh Teaching Award. And ensuring this is am embedded process, with connection to processes, language, etc. with registry, board of studies, etc. And also with multiple facilitators.

I looked for frameworks and focused on three to evaluate. These tend to be theoretical, and don’t always work in practice. After trying those all out we found CAIeRO works best, focusing on designing learning experiences over development of content, structured format of the two day workshop. And it combines pedagogy, technology, learner experience.

We have developed the CAIeRO into a slightly different form, the ELDeR Framework, with the addition of assessment and feedback.

Finally! Theory and Practice – Ruth McQuillan, Co-Programme Director, Master of Public Health (online)

Prior to the new MPH programme I have been working in online learning since 2011. I am part of a bigger team – Christine Matthews is our learning technologist and we have others who have come on board for our new programme. Because we had a new programme launching we were very keen to be part of it. So I’m going to talk about how this worked, how we felt about it, etc.

We launched the online MPG in September 2015, which involved developing lots of new courses but also modifying lots of existing courses. And we have a lot of new staff so we wanted to give a sense of building a new team – as well as learning for ourselves how to do it all properly.

So, the stages of the workshop we went through should give you a sense of it. I’ve been on lots of courses and workshops where you learn about something but you don’t have the practical application. And then you have a course to prepare in practice, maybe without that support. So having both aspects together was really good and helpful.

The course we were designing was for mid career professionals from across the world. We were split into two teams – with each having a blend of the kinds of people Fiona talked about – programme team and colleagues from IS and elsewhere. We both developed programme and course mission statements as a group, then compared and happily those were quite close, we reached consensus and that really felt like we were pulling together as a team. And we also checked the course for consistency with the programme.

Next, we looked at the look and feel aspects. We used cards that were relevant for our course, using workshop cards and post it notes, rejecting non relevant cards, using our choice of the cards and some of our own additions.

So, Fiona talked about beginning with the end in mind, and we tried to do that. We started by thinking about what we wanted our students to be able to do at the end of the course. That is important as this is a professional course where we want to build skills and understanding. So, we wanted to focus on what they should know at the end of the course, and only then look at the knowledge they would need. And that was quite a different liberating approach.

And at this point we looked at the SCQF level descriptors to think about learning outcomes, the “On completion of this course you will be able to…” I’m not sure we’d appreciated the value and importance of our learning outcomes before, but actually in the end this was one of the most useful parts of the process. We looked for Sense (are they clear to the learner); Level (are they appropriate to the level of module); Accessibility (are they accessible).

And then we needed to think about assessment and alignment, looking at how we would assess the course, how this fitted into the bigger picture etc.

The next step was to storyboard the course. And by the end of Day One we had a five week course and a sixth week for assessment, we has learning outcomes and how they’d be addressed, assessment, learning activities, concerns, scaffolding. And we thought we’d done a great job! We came back on day two and when we came back we spend maybe half a day recapping, changing… Even if you can’t do a 2 day workshop at least try to do two half days with a big gap between/overnight as we found that space away very helpful.

And once finalised we built a prototype online. And we had a reality check from a critical friend, which was very helpful. We reviewed and adjusted and then made a really detailed action plan. That plan was really helpful.

Now, at the outside we were told that we could come into this process at any point. We had quite a significantly complete idea already and that helped us get real value from this process.

So, how did it feel and what did we learn? Well it was great to have a plan, to see the different areas coming together. The struggle was difficult but important, and it was excellent for team building. “To learn and not to do is really not to learn. To do and not to learn is really not to know. And actually at the end of the day we were really enthusiastic about the process and it was really good to see that process, to put theory into practice, and to do this all in a truly collaborative experience.

How has it changed us? Well we are putting all our new courses through this process. We want to put all our existing courses through this process. We involved more people in the process, in different roles and stages, including students where we can. And we have modified the structure.

Q&A

Q1) Did you go away to do this?

A1) Yes, we went to Dovecot Gallery on Infirmary Street.

A1 – FH) I had some money to do that but I wasn’t kidding that a new space and nice food is important. We are strict on you being there, or not. We expect full on participation. So for those going forward we are looking at rooms in other places – in Evolution House, or in Moray House, etc. Somewhere away from normal offices etc. It has to be a focused. And the value of that is huge, the time up front is really valuable.

A1 – RM) It is also really important for understanding what colleagues are doing, which helps ensure the coherence of the programme, and it is really beneficial to the programme.

Q2) Dow different do you think your design ended up if you hadn’t done this?

A2 – RM) I think one of my colleagues was saying today that she was gently nudged by colleagues to avoid mistakes or pitfalls, to not overload the course, to ensure coherence, etc. I think it’s completely different to how it would have been. And also there were resources and activities – lectures and materials – that could be shared where gaps were recognised.

A2 – FH) If this had been content driven it would be hard as a facilitator. But thinking about the structure, the needs, the learner experience, that can be done, with content and expertise already being brought into that process. It saves time in the long run.

A2 – RM) I know in the past when I’ve been designing courses you can find that you put activities in a particular place without purpose, to make sure there is an activity there… But this process helped keep things clear, coherent and to ensure any activity is clearly linked to a learning outcome, etc.

Q3) Once you’d created the learning outcomes, did you go back and change any of theme?

A3 – FH) On Day 2 there was something that wasn’t quite right…

A3 – RM) It was something too big for the course, and we needed to work that through. The course we were working on in February and that will run for the first time in the new academic year. But actually the UoE system dictates that learning outcomes should be published many months/more than a year in advance. So with new courses we did ask the board of studies if we could provide the learning outcomes to them later on, once defined. They were fine.

A3 – FH) That is a major change that we are working on. But not all departments run the same process or timetable.

A3 – RM) Luckily our board of studies were very open to this, it was great.

Q4) Was there any focus on student interaction and engagement in these process.

A4 – FH) It was part of those cards early in the process, it is part of the design work. And that stage of the cards, the consensus building, those are huge collaborative and valuable sessions.

Q5) And how did you support/require that?

A5 – FH) In that storyboard you will see various (yellow) post its showing assessment and feedback wove in across the course, ensuring the courses you design really do align with that wider University strategy.

Learning Design: Paying It Forward – Christina Matthews

There is a shift across the uni to richer approaches.

I’m going to talk about getting learning technologist involved and why that matters.

The LT can inform the process in useful and creative ways. They can bring insights into particular tools, affordances, and ways to afford or constrain the behaviours of students. They also have a feel for digital literacy of students, as well as being able to provide some continuity across the course in terms of approaches and tools. And having LT in the design process, academic staff can feel supported and better able to take risks and do new things. And the LT can help that nothing is lost between the design workshop, and the actual online course and implementation.

So, how are we paying this forward? Well we are planning learning design workshops for all our new courses for 2015-16 and 2016-17. We really did feel the benefits of 2 days but we didn’t think it was going to be feasible for all of our teams. We felt that we needed to adapt the workshop to fit into one day, so we will be running these as one day workshops and we have prioritised particular aspects to enable that.

The two day workshop format for CAIeRO follows several stages:

  • Stage 1: Course blueprint (mission, learning outcomes, assessment and feedback)
  • Stage 2: Storyboarding
  • Stage 3: Rapid prototyping in the VLE
  • Stage 4: Critical friend evaluation of VLE prototype
  • Stage 5: adjust and review from feedback
  • Stage 6: Creating an action plan
  • Stage 7: reflecting on the workshop in relation to the UK Professional Standards Framework.
  • For the one day workshop we felt the blue print (1), storyboard (2) and action plan stages (6) were essential. The prototyping can be done afterwards and separately, although it is a shame to do that of course.

So, we are reviewing and formalising our 1 day workshop model, which may be useful elsewhere. And we are using these approaches for all the courses on our programme, including new and existing courses. And we are very much looking forward to the ELDeR (Edinburgh Learning Design Roadmap).

Q&A

Q1) When you say “all” programmes, do you mean online or on-campus programmes?

A1) Initially the online courses but we have a campus programme that we really want to connect up, to make the courses more blended, so I think it will feed into our on campus courses. A lot of our online tutors teach both online and on campus, so that will also lead some feeding in here.

Q2) How many do you take to the workshop?

A2) You can have quite a few. We’ve had programme director, course leader, learning technologist, critical friends, etc.

A2 – FH) There are no observers in the room for workshops – lots are wanting to understand that. There are no observers in the room, you have to facilitate the learning objectives section very carefully. Too many people is not useful. Everyone has to be trusted, they have to be part of the process. You need a support librarian, the learning technologist has to squarely be part of the design, student, reality checker, QA… I’ve done at most 8 people. In terms of students you need to be able to open and raw…. So, is it OK to have students in the room… Some conversations being had may not be right for that co-creation type idea. Maybe alumni are better in some cases. Some schools don’t have their own learning technologist, so we bring one. Some don’t have a VLE, so we bring one they can play with.

A2 – CM) In the pilot there were 8 in some, but it didn’t feel like too many in the room.

Q3) As a learning technologist have the workshops helped your work?

A3 – CM) Yes, hugely. That action plan really maps out every stage very clearly. Things can come in last minute and all at the same time otherwise, so that is great. And when big things are agreed in the workshop, you can then focus on the details.

A3 – FH) We are trying to show how actually getting this all resolved up front actually saves money and time later on, as everything is agreed.

Q4) Thinking way ahead… People will do great things… So if we have the course all mapped out here, and well agreed, what happens when teams change – how do you capture and communicate this. Should you have a mini reprise of this to revisit it? How does it go over the long term?

A4 – FH) That’s really true. Also if technologist isn’t the one delivering it, that can also be helpful.

A4 – CM) One thing that comes out of this is a CAIeRO planner that can be edited and shared, but yes, maybe you revisit it for future staff…

A4 – FH) Something about ownership of activities, to give the person coming in and feel ownership. And see how it works before and afterwards. Pointing them to document, to output of storyboard, to get ownership. That’s key to facilitation too.

Q4) So, you can revisit activities etc. to achieve Learning outcome…

A4 – FH) That identification of learning outcomes are clear in the storyboards and documents.

Q5) How often do you meet and review programmes? Every 2 years, every 5 years?

A5 – FH) You should review every 5 years for PG.

Comment) We have an annual event, see what’s working and what isn’t and that is very very valuable and helpful. But that’s perhaps unusual.

A5 – FH) That’s the issue of last minute or isolated activities. This process is a good structure for looking at programme and course. Clearly programme has assessment across it so even though we are looking at the course here, it has that consistency. With any luck we can get this stuff embedded in board of studies etc.

A5 – RM) For us doing this process also changed us.

A5 – FH) That report is huge but the universities I looked at these processes are mandatory not optional. But mandatory can make things more about box ticking in some ways…

Learning Design: 6 Months on – Meredith Corey, School of Education 

We are developing a pilot UG course in GeoSciences and Education collaboration, Sustainability and Social Responsibility, running 2016/17. We are 2 online learning educators working from August 2015 to April 2016. This is the first online level 8 course for on-campus students. And there are plans to adapt the course for the wider community – including staff, alumni etc.

So in the three months before the CAIeRO session, we had started looking at existing resources, building a course team, investigating VLEs. The programme is on sustainability. We looked into types of resources and activities. And we had started drafting learning outcomes and topic storyboarding, with support from Louise Connelly who was (then) in IAD.

So the workshop was a 2 day event and we began with the blueprinting. We had similar ideas and very different ways to describe them so, what was very useful for us, was finding common language and ways to describe what we were doing. We didn’t drastically change our learning outcomes, but lots of debate about the wording. Trying to ensure the learning outcomes were appropriate for level 8 SCQF levels, trying not to overload them. And this whole process has helped us focus on our priorities, our vocabulary, the justification and clear purpose.

The remainder of the workshop was spent on storyboarding. We thought we were really organised in terms of content, videos, etc. But actually that storyboarding, after that discussion of priorities, was really useful. Our storyboard generated three huge A0 sheets to understand the content, the ways students would achieve the learning outcomes. It is an online course and there are things you don’t think about but need to consider – how do they navigate the course? How do they find what they need? How do they find what they need? And Fiona and colleagues were great for questioning and probing that.

We did some prototyping but didn’t have time for reality checks – but we have that process lined up for our pilot in the summer. We also took that storyboard and transferred that information to a huge Popplet that allowed us to look at how the feedback and feed forward fits into the course; how we could make that make sense across the course – it’s easy to miss that feedback and feed forward is too late when you are looking week by week.

The key CAIeRO benefits for us were around exploring priorities (and how these may differ for different cohorts); it challenged our assumptions; it formalised our process and this is useful for future projects; focused on all learners and their experience; and really helped us understand our purpose here. And coming soon we shall return to the Popplet to think about the wider community.

Q&A

Q1) I know with one course the head of school was concerned that an online programme might challenge the value of the face to face, or the concern of replacing the face to face course, and how that fits together.

A1) The hope with this course is that the strength is that it brings together students from as many different schools as possible, to really deal with timetabling barriers, to mix students between schools. It would be good if both exists to complement in each others.

A1 – FH) Its not intended as a replacement… In this course’s mission statement for this, it plays up interdisciplinary issues, and that includes use of OERs, reuse, etc. And talking about doing this stuff.

A1) And also the idea is to give students a great online learning experience that means they might go on and do online masters programmes. And hopefully include staff and alumni that also help that mix, that interdisciplinary thing.

Q2) Do you include student expectations in this course? What about student backgrounds?

A2) We have tried to ensure that tutorial groups play to student strengths and interests, making combinations across schools. We are trialling the course with evaluation through very specific questions.

A2 – FH) And there will assessment that asks students to place that learning into their own context, location, etc.

Course Design and your VLE – Ross Ward

I want to talk quickly about how you translate a storyboard into your VLE, in very general terms. Taking your big ideas and making them a course. One thing I like to talk about a lot is user experience – you only need one back experience in Learn or Moodle to really put you off. So you really need to think about ensuring the experience of the VLE and the experience of the course all need to fit together. How you manage or use your VLE is up to do. Once you know what you want to do, you can then pick your technology, fitting your needs. And you’ll need a mix of content, tools, activities, grades, feedback, guidance. If you are an ODL student how you structure that will be very very important, if blended it’s still important. You don’t need your VLE to be a filing cabinet, it can be much more. But it also doesn’t have to be a grand immersive environment, you need it to fit your needs appropriately. And the VLE experience should reflect the overall course experience.

When you have that idea of purpose, you hit the technology and you have kind of a blank canvas. It’s a bit Mona Lisa by numbers… The tools are there but there are easier ways to make your course better. The learning design idea of the storyboard and the user experience of the course context can be very helpful. That is really useful for ensuring students understand what they are doing, creating a digital version of your course, and understanding where you are right now as a student. Arguably a good VLE user experience is one where you could find what you are looking for without any prior knowledge of the course… We get many support calls from those simply looking for information. You may have some pre-requisite stuff, but you need to really make everything easy.

Navigation is key! You need menus. You need context links. You need suggested link. You want to minimise the number of clicks and complexity.

Remember that you should present your material for online, not like a textbook. Use sensible headings. Think about structure. And test it out – ask a colleague, as a student, ask LTW.

And think about consistency – that will help ensure that you can build familiarity with approach, consistently presenting your programme/school brand and look and feel, perhaps also template.

We know this is all important, and we want to provide more opportunity to support that, with examples and resources to draw upon!

Closing Fiona Hale

Huge thanks to Ross for organising today. Huge thanks to our speakers today!

If you are interested in this work do find me at the end, do come talk to me. We have workshops coming up – ELDeR workshop evaluations – and there we’ll talk about design challenges and concerns. That might be learning analytics – and thinking about pace and workshops. For all of these we are addressing particular design challenges – the workshop can concertina to that. There is no rule about how long things take – and whether one day or two days is the number, but sometimes one won’t be enough.

I would say for students it’s worth thinking about sharing the storyboards, the assessment and feedback and reasons for it, so that they understand it.

We go into service in June and July, with facilitators across the schools. Do email me with questions, to offer yourselves as facilitators.

Thank you to all of our University colleagues who took part in this really interesting session!

You can read much more about Edinburgh Learning Design roadmap – and read the full scoping report – on the University of Edinburgh Learning Design Service website. 

Share/Bookmark

A Summer of New Digital Footprints…

Posted on by

It has been a while since I’ve posted something other than a liveblog here but it has been a busy summer so it seems like a good time to share some updates…

A Growing Digital Footprint

Last September I was awarded some University of Edinburgh IS Innovation Fund support to develop a pilot training and consultancy service to build upon the approaches and findings of our recent PTAS-funded Managing Your Digital Footprint research project.

During that University of Edinburgh-wide research and parallel awareness-raising campaign we (my colleague – and Digital Footprint research project PI – Louise Connelly of IAD/Vet School, myself, and colleagues across the University) sought to inform students of the importance of digital tracks and traces in general, particularly around employment and “eProfessionalism”. This included best practice advice around use of social media, personal safety and information security choices, and thoughtful approaches to digital identity and online presences. Throughout the project we were approached by organisations outside of the University for similar training, advice, and consulting around social media best practices and that is how the idea for this pilot service began to take shape.

Over the last few months I have been busy developing the pilot, which has involved getting out and about delivering social media training sessions for clients including NHS Greater Glasgow and Clyde (with Jennifer Jones); for the British HIV Association (BHIVA) with the British Association for Sexual Health and HIV (BASHH) (also with Jennifer Jones); developing a “Making an Impact with your Blog” Know How session for the lovely members of Culture Republic; leading a public engagement session for the very international gang at EuroStemCell, and an “Engaging with the Real World” session for the inspiring postgrads attending the Scottish Graduate School of Social Science Summer School 2016. I have also been commissioned by colleagues in the College of Arts, Humanities and Social Sciences to create an Impact of Social Media session and accompanying resources (the latter of which will continue to develop over time). You can find resources and information from most of these sessions over on my presentations and publications page.

These have been really interesting opportunities and I’m excited to see how this work progresses. If you do have an interest in social media best practice, including advice for your organisation’s social media practice, developing your online profile, or managing your digital footprint, please do get in touch and/or pass on my contact details. I am in the process of writing up the pilot and looking at ways myself and my colleagues can share our expertise and advice in this area.

Adventures in MOOCs and Yik Yak

So, what next?

Well, the Managing Your Digital Footprint team have joined up with colleagues in the Language Technology Group in the School of Informatics for a new project looking at Yik Yak. You can read more about the project, “A Live Pulse: Yik Yak for Understanding Teaching, Learning and Assessment at Edinburgh“, on the Digital Education Research Centre website. We are really excited to explore Yik Yak’s use in more depth as it is one of a range of “anonymous” social networking spaces that appear to be emerging as important alternative spaces for discussion as mainstream social media spaces lose favour/become too well inhabited by extended families, older contacts, etc.

Our core Managing Your Digital Footprint research also continues… I presented a paper, co-written with Louise Connelly, at the European Conference on Social Media 2016 this July on “Students’ Digital Footprints: curation of online presences, privacy and peer supportâ€�. This summer we also hosted visiting scholar Rachel Buchanan of University of Newcastle, Australia who has been leading some very interesting work into digital footprints across Australia. We are very much looking forward to collaborating with Rachel in the future – watch this space!

And, more exciting news: my lovely colleague Louise Connelly (University of Edinburgh Vet School) and I have been developing a Digital Footprint MOOC which will go live later this year. The MOOC will complement our ongoing University of Edinburgh service (run by IAD) and external consultancy word (led by us in EDINA) and You can find out much more about that in this poster, presented at the European Conference on Social Media 2016, earlier this month…

Preview of Digital Footprint MOOC Poster

Alternatively, you could join me for my Cabaret of Dangerous Ideas 2016 show….

Cabaret of Dangerous Ideas 2016 - If I Googled You, What Would I Find? Poster

The Cabaret of Dangerous Ideas runs throughout the Edinburgh Fringe Festival but every performance is different! Each day academics and researchers share their work by proposing a dangerous idea, a provocative question, or a challenge, and the audience are invited to respond, discuss, ask difficult questions, etc. It’s a really fun show to see and to be part of – I’ve now been fortunate enough to be involved each year since it started in 2013. You can see a short video on #codi2016 here:

In this year’s show I’ll be talking about some of those core ideas around managing your digital footprint, understanding your online tracks and traces, and reflecting on the type of identity you want to portray online. You can find out more about my show, If I Googled You What Would I Find, in my recent “25 Days of CODI” blog post:

25 Days of CoDI: Day 18

You’ll also find a short promo film for the series of data, identity, and surveillance shows at #codi2016 here:

So… A very busy summer of social media, digital footprints, and exciting new opportunities. Do look out for more news on the MOOC, the YikYak work and the Digital Footprint Training and Consultancy service over the coming weeks and months. And, if you are in Edinburgh this summer, I hope to see you on the 21st at the Stand in the Square!

 

Share/Bookmark

Principal’s Teaching Award Scheme Forum 2016 – Liveblog

Posted on by

Today I’m at the University of Edinburgh Principal’s Teaching Award Scheme Forum 2016: Rethinking Learning and Teaching Together, an event that brings together teaching staff, learning technologists and education researchers to share experience and be inspired to try new things and to embed best practice in their teaching activities.

I’m here partly as my colleague Louise Connelly (Vet School, formerly of IAD) will be presenting our PTAS-funded Managing Your Digital Footprint project this afternoon. We’ll be reporting back on the research, on the campaign, and on upcoming Digital Foorprints work including our forthcoming Digital Footprint MOOC (more information to follow) and our recently funded (again by PTAS) project: “A Live Pulse: YikYak for Understanding Teaching, Learning and Assessment at Edinburgh.

As usual, this is a liveblog so corrections, comments, etc. welcome. 

Velda McCune, Deputy Director of the IAD who heads up the learning and teaching team, is introducing today:

Welcome, it’s great to see you all here today. Many of you will already know about the Principal’s Teaching Award Scheme. We have funding of around £100k from the Development fund every year, since 2007, in order to look at teaching and learning – changing behaviours, understanding how students learn, investigating new education tools and technologies. We are very lucky to have this funding available. We have had over 300 members of staff involved and, increasingly, we have students as partners in PTAS projects. If you haven’t already put a bid in we have rounds coming up in September and March. And we try to encourage people, and will give you feedback and support and you can resubmit after that too. We also have small PTAS grants as well for those who haven’t applied before and want to try it out.

I am very excited to welcome our opening keynote, Paul Ashwin of Lancaster University, to kick off what I think will be a really interesting day!

Why would going to university change anyone? The challenges of capturing the transformative power of undergraduate degrees in comparisons of quality  – Professor Paul Ashwin

What I’m going to talk about is this idea of undergraduate degrees being transformative, and how as we move towards greater analytics, how we might measure that. And whilst metrics are flawed, we can’t just ignore these. This presentation is heavily informed by Lee Schumers work on Pedagogical Content Knowledge, which always sees teaching in context, and in the context of particular students and settings.

People often talk about the transformative nature of what their students experience. David Watson was, for a long time, the President for the Society of Higher Education (?) and in his presidential lectures he would talk about the need to be as hard on ourselves as we would be on others, on policy makers, on decision makers… He said that if we are talking about education as educational, we have to ask ourselves how and why this transformation takes place; whether it is a planned transformation; whether higher education is a nesseccary and/or sufficient condition for such transformations; whether all forms of higher education result in this transformation. We all think of transformation as important… But I haven’t really evidenced that view…

The Yerevan Communique: May 2015 talks about wanting to achieve, by 2020, a European Higher Education area where there are common goals, where there is automatic recognition of qualifictions and students and graduates can move easily through – what I would characterise is where Bologna begins. The Communique talks about higher education contributing effectively to build inclusive societies, found on democratic values and human rights where educational opportunities are part of European Citizenship. And ending in a statement that should be a “wow!” moment, valuing teaching and learning. But for me there is a tension: the comparability of undergraduate degrees is in conflict with the idea of transformational potential of undergraduate degrees…

Now, critique is too easy, we have to suggest alternative ways to approach these things. We need to suggest alternatives, to explain the importance of transformation – if that’s what we value – and I’ll be talking a bit about what I think is important.

Working with colleagues at Bath and Nottingham I have been working on a project, the Pedagogic Quality and Inequality Project, looking at Sociology students and the idea of transformation at 2 top ranked (for sociology) and 2 bottom ranked (for sociology) universities and gathered data and information on the students experience and change. We found that league tables told you nothing about the actual quality of experience. We found that the transformational nature of undergraduate degrees lies in changes in students sense of self through their engagement with discplinary knowledge. Students relating their personal projects to their disciplines and the world and seeing themselves implicated in knowledge. But it doesn’t always happen – it requires students to be intellectually engaged with their courses to be transformed by it.

To quote a student: “There is no destination with this discipline… There is always something further and there is no point where you can stop and say “I understaood, I am a sociologist”… The thing is sociology makes you aware of every decision you make: how that would impact on my life and everything else…” And we found the students all reflecting that this idea of transformation was complex – there were gains but also losses. Now you could say that this is just the nature of sociology…

We looked at a range of disciplines, studies of them, and also how we would define that in several ways: the least inclusive account; the “watershed” account – the institutional type of view; and the most inclusive account. Mathematics has the most rich studies in this area (Wood et al 2012) where the least inclusive account is “Numbers”, watershed is “Models”, most inclusive is “approach to life”. Similarly Accountancy moves from routine work to moral work; Law from content to extension of self; Music from instrument to communicating; Geograpy is from general world to interactions; Geoscience is from composition of earth – the earth, to relations earth and society. Clearly these are not all the same direction, but they are accents and flavours of the same time. We are going to do a comparison next year on chemistry and chemical engineering, in the UK and South Africa, and actually this work points at what is particular to Higher Education being about engaging with a system of knowledge. Now, my colleague Monica McLean would ask why that’s limited to Higher Education, couldn’t it apply to all education? And that’s valid but I’m going to ignore it just for now!

Another students comments on transformation of all types, for example from wearing a tracksuit to lectures, to not beginning to present themselves this way. Now that has nothing to do with the curriculum, this is about other areas of life. This student almost dropped out but the Afro Carribean society supported and enabled her to continue and progress through her degree. I have worked in HE and FE and the way students talk about that transformation is pretty similar.

So, why would going to university change anyone? It’s about exposure to a system of knowledge changing your view of self, and of the world. Many years ago an academic asked what the point of going to university was, given that much information they learn will be out of date. And the counter argument there is that engagement with seeing different perspectives, to see the world as a sociologist, to see the world as a geographer, etc.

So, to come back to this tension around the comparability of undergraduate degrees, and the transformational potential of undergraduate degrees. If we are about transformation, how do we measure it? What are the metrics for this? I’m not suggesting those will particularly be helpful… But we can’t leave metrics to what is easy to gather, we have to also look at what is important.

So if we think of the first area of compatibility we tend to use rankings. National and international higher education rankings are a dominant way of comparing institutions’ contributions to student success. All universities have a set of figures that do them well. They have huge power as they travel across a number of contexts and audiences – vice chancellors, students, departmental staff. It moves context, it’s portable and durable. It’s nonsense but the strength of these metrics is hard to combat. They tend to involved unrelated and incomparable measures. Their stability reinforces privilege – higher status institutions tend to enrol a much greated proportion of privileged students. You can have some unexpected outcomes but you have to have Oxford, Cambridge, Edinburgh, UCL, Imperial all near the top then your league table is rubbish… Because we already know they are the good universities… Or at least those rankings reinforce the privilege that already exists, the expectations that are set. They tell us nothing about transformation of students. But are skillful performances shaped by generic skills or students understanding of a particular task and their interactions with other people and things?

Now the OECD has put together a ranking concept on graduate outcomes, the AHELO, which uses tests for e.g. physics and engineering – not surprising choices as they have quite international consistency, they are measurable. And they then look at generic tests – e.g a deformed fish is found in a lake, using various press releases and science reports write a memo for policy makers. Is that generic? In what way? Students doing these tests are volunteers, which may not be at all representative. Are the skills generic? Education is about applying a way of thinking in an unstructured space, in a space without context. Now, the students are given context in these texts so it’s not a generic test. But we must be careful about what we measure as what we measure can become an index of quality or success, whether or not that is actually what we’d want to mark up as success. We have strategic students who want to know what counts… And that’s ok as long as the assessment is appropriately designed and set up… The same is true of measures of success and metrics of quality and teaching and learning. That is why I am concerned by AHELO but it keeps coming back again…

Now, I have no issue with the legitimate need for comparison, but I also have a need to understand what comparisons represent, how they distort. Are there ways to take account of students’ transformation in higher education?

I’ve been working, with Rachel Sweetman at University of Oslo, on some key characteristics of valid metrics of teaching quality. For us reliability is much much more important than availability. So, we need ways to assess teaching quality that:

  • are measures of the quality of teaching offered by institutions rather than measures of institutional prestige (e.g. entry grades)
  • require improvements in teaching practices in order to improve performance on the measures
  • as a whole form a coherent set of metrics rather than a set of disparate measures
  • are based on established research evidence about high quality teaching and learning in higher education
  • reflect the purposes of higher education.

We have to be very aware of Goodhearts’ rule that we must be wary of any measure that becomes a performance indicator.

I am not someone with a big issue with the National Student Survey – it is grounded in the right things but the issue is that it is run each year, and the data is used in unhelpful distorted ways – rather than acknowledging and working on feedback it is distorting. Universities feel the need to label engagement as “feedback moments” as they assume a less good score means students just don’t understand when they have that feedback moment.

Now, in England we have the prospect of the Teaching Excellence Framework English White Paper and Technical Consultation. I don’t think it’s that bad as a prospect. It will include students views of teaching, assessment and academic support from the National Student Survey, non completion rates, measures over three years etc. It’s not bad. Some of these measures are about quality, and there is some coherence. But this work is not based on established research evidence… There was great work here at Edinburgh on students learning experiences in UK HE, none of that work is reflected in TEF. If you were being cynical you could think they have looked at available evidence and just selected the more robust metrics.

My big issue with Year 2 TEF metrics are how and why these metrics have been selected. You need a proper consultation on measures, rather than using the White Paper and Technical Consultation to do that. The Office for National Statistics looked at measures and found them robust but noted that the differences between institutions scores on the selected metrics tend to be small and not significant. Not robust enough to inform future work according to the ONS. It seems likely that peer review will end up being how we differentiate between institution.

And there are real issues with TEF Future Metrics… This comes from a place of technical optimism that if you just had the right measures you’d know… This measure ties learner information to tax records for “Longitudinal Education Outcomes data set” and “teaching intensity”. Teaching intensity is essentially contact hours… that’s game-able… And how on earth is that about transformation, it’s not a useful measure of that. Unused office hours aren’t useful, optional seminars aren’t useful…  Keith Chigwell told me about a lecturer he knew who lectured a subject, each week fewer and fewer students came along. The last three lectures had no students there… He still gave them… That’s contact hours that count on paper but isn’t useful. That sort of measure seems to come more from ministerial dinner parties than from evidence.

But there are things that do matter… There is no mechanism outlines for a sector-wide discussion of the development of future metrics. What about expert teaching? What about students relations to knowledge? What about the first year experience – we know that that is crucial for student outcomes? Now the measures may not be easy, but they matter. And what we also see is the Learning Gains project, but they decided to work generically, but that also means you don’t understand students particular engagement with knowledge and engagement. In generic tests the description of what you can do ends up more important than what you actually do. You are asking for claims for what they can do, rather than performing those things. You can see why it is attractive, but it’s meaningless, it’s not a good measure of what Higher Education can do.

So, to finish, I’ve tried to put teaching at the centre of what we do. Teaching is a local achievement – it always shifts according to who the students are , what the setting is, and what the knowledge is. But that also always makes it hard to capture and measure. So what you probably need is a lot of different imperfect measures that can be compared and understood as a whole. However, if we don’t try we allow distorting measures, which reinforce inequalities, to dominate. Sometimes the only thing worse than not being listened to by policy makers, is being listened to them. That’s when we see a Frankenstein’s Monster emerge, and that’s why we need to recognise the issues, to ensure we are part of the debate. If we don’t try to develop alternative measures we leave it open to others to define.

Q&A

Q1) I thought that was really interesting. In your discussion of transformation of undergraduate students I was wondering how that relates to less traditional students, particularly mature students, even those who’ve taken a year out, where those transitions into adulthood are going to be in a different place and perhaps where critical thinking etc. skills may be more developed/different.

A1) One of the studies I talked about was London Metropolitan University has a large percentage of mature students… And actually there the interactions with knowledge really did prove transformative… Often students lived at home with family whether young or mature students. That transformation was very high. And it was unrelated to achievements. So some came in who had quite profound challenges and they had transformation there. But you have to be really careful about not suggesting different measures for different students… That’s dangerous… But that transformation was there. There is lots of research that’s out there… But how do we transform that into something that has purchase… recognising there will be flaws and compromises, but ensuring that voice in the debate. That it isn’t politicians owning that debate, that transformations of students and the real meaning of education is part of that.

Q2) I found the idea of transformation that you started with really interesting. I work in African studies and we work a lot on colonial issues, and of the need to transform academia to be more representative. And I was concerned about the idea of transformation as a colonial type issue, of being like us, of dressing like that… As much as we want to challenge students we also need to take on and be aware of the biases inherent in our own ways of doing things as British or Global academics.

A2) I think that’s a really important question. My position is that students come into Higher Education for something. Students in South Africa – and I have several projects there – who have nowhere to live, have very little, who come into Higher Education to gain powerful knowledge. If we don’t have access to a body of knowledge, that we can help students gain access to and to gain further knowledge, then why are we there? Why would students waste time talking to me if I don’t have knowledge. The world exceeds our ability to know it, we have to simplify the world. What we offer undergraduates is powerful simplifications, to enable them to do things. That’s why they come to us and why they see value. They bring their own biographies, contexts, settings. The project I talked about is based in the work of Basil Bernstein who argues that the knowledge we produce in primary research… But when we design curriculum it isn’t that – we engage with colleagues, with peers, with industry… It is transformed, changed… And students also transform that knowledge, they relate it to their situation, to their own work. But we are only a valid part of that process if we have something to offer. And for us I would argue it’s the access to body of knowledge. I think if we only offer process, we are empty.

Q3) You talked about learning analytics, and the issues of AHELO, and the idea of if you see the analytics, you understand it all… And that concept not being true. But I would argue that when we look at teaching quality, and a focus on content and content giving, that positions us as gatekeepers and that is problematic.

A3) I don’t see knowledge as content. It is about ways of thinking… But it always has an object. One of the issues with the debate on teaching and learning in higher education is the loss of the idea of content and context. You don’t foreground the content, but you have to remember it is there, it is the vehicle through which students gain access to powerful ways of thinking.

Q4) I really enjoyed that and I think you may have answered my question.. But coming back to metrics you’ve very much stayed in the discipline-based silos and I just wondered how we can support students to move beyond those silos, how we measure that, and how to make that work.

A4) I’m more course than discipline focused. With the first year of TEF the idea of assessing quality across a whole institution is very problematic, it’s programme level we need to look at. inter-professional, interdisciplinary work is key… But one of the issues here is that it can be implied that that gives you more… I would argue that that gives you differently… It’s another new way of seeing things. But I am nervous of institutions, funders etc. who want to see interdisciplinary work as key. Sometimes it is the right approach, but it depends on the problem at hand. All approaches are limited and flawed, we need to find the one that works for a given context. So, I sort of agree but worry about the evangelical position that can be taken on interdisciplinary work which is often actually multidisciplinary in nature – working with others not genuinely working in an interdisciplinary way.

Q5) I think to date we focus on objective academic ideas of what is needed, without asking students what they need. You have also focused on the undergraduate sector, but how applicable to the post graduate sector?

A5) I would entirely agree with your comment. That’s why pedagogic content matters so much. You have to understand your students first, as well as then also understanding this body of knowledge. It isn’t about being student-centered but understanding students and context and that body of knowledge. In terms of your question I think there is a lot of applicability for PGT. For PhD students things are very different – you don’t have a body of knowledge to share in the same way, that is much more about process. Our department is all PhD only and there process is central. That process is quite different at that level… It’s about contributing in an original way to that body of knowledge as its core purpose. That doesn’t mean students at other levels can’t contribute, it just isn’t the core purpose in the same way.

And with that we are moving to coffee… The rest of the programme for the day is shown below, updates to follow all day. 

11.50-12.35 Parallel Sessions from PTAS projects

12.35 – 13.35 Lunch and informal discussion

13:35 -14.20 Parallel Sessions from PTAS projects

14.20-15.00 Refreshments and networking

15.00-16.00 Closing Keynote : Helen Walker, GreyBox Consulting and Bright Tribe Trust

16:00-16.30 Feedback and depart

Share/Bookmark

Belated Liveblog: eLearning@ed 2016

Posted on by

Last week I was delighted to be part of the team organising the annual eLearning@ed Conference 2016. The event is one of multiple events and activities run by and for the eLearning@ed Forum, a community of learning technologists, academics, and those working with learning technologies across the University of Edinburgh. I have been Convener of the group since last summer so this was my first conference in this role – usually I’m along as a punter. So, this liveblog is a little later than usual as I was rather busy on the day…

Before going into my notes I do also want to say a huge thank you to all who spoke at the event, all who attended, and an extra special thank you to the eLearning@ed Committee and Vlad, our support at IAD. I was really pleased with how the event went – and feedback has been good – and that is a testament to the wonderful community I have the privilege of working with all year round here at Edinburgh.

Note: Although I have had a chance to edit these notes they were taken live so just let me know if you spot any errors and I will be very happy to make any corrections. 

The day opened with a brief introduction from me. Obviously I didn’t blog this but it was a mixture of practical information, enthusiasm for our programme, and an introduction to our first speaker, Melissa Highton:

Continue reading →

Upcoming Events in Innovative Learning Week 2016

Posted on by

For the last few years the University of Edinburgh have run an “Innovative Learning Week” in which no traditional lectures or tutorials take place, instead students (and staff) are encouraged to experiment, to engage in new ways, to participate in events and teaching activities beyond their usual discipline or subject areas. It is a really lovely concept and I am always amazed at the range of events and collaborations that take place in that very busy week.

This year Innovative Learning Week runs from Monday 15th to Friday 19th February and I am involved in a few events that I thought I would share here for those based at Edinburgh (do sign up!) and for the interest of others who may be curious about what an ILW event looks like…

History of Medicine Wikipedia Editathon

This event, a follow up last year’s very successful editathon, is something I have been involved in the planning of (and will be baking for) although I’ll only be able to be there on the Thursday. However, a fantastic group of information services, academic and Wikipedian in Residence folks are making this event happen and it should be both fun and really interesting. Great for those wanting to brush up their Wikipedia skills too. 

Join the Innovative Learning Week History of Medicine Wikipedia Editathon (open to students, staff, and all others who are interested), where you will have an opportunity to edit Wikipedia and meet our new Wikimedian in Residence, Ewan McAndrew. Join us in re-writing the Wikipedia pages of Edinburgh’s infamous medical figures including body-snatcher William Burke, the intriguing Dr. James Miranda Barry, or choose to enhance and create content for notable University of Edinburgh alumni (see the list under the How do I prepare section http://bit.ly/ILWEditathonEventPage).

Wikipedia training provides staff valuable digital skills to support CPD as well as hands on experience using an open access educational repository. No experience necessary as each session will offer Wikipedia editing and publishing training and the opportunity to observe online collaboration, public engagement, knowledge exchange, and scholarly communication in action.

Join in for one session, a full day, or for all three (sessions run in David Hume Tower, Teaching Studio LG.07):

  • TUESDAY 16                       Session1: 2pm-5pm
  • WEDNESDAY 17                S2: 10am-1pm; S3: 2-5pm
  • THURSDAY 18                    S4: 10am-1pm; S5: 2-5pm

Sign up: http://bit.ly/ILWEditathon2016 and/or follow us and share on Twitter: #ILWEditathon @LTW_UOE. If you are attending please bring your own personal laptop or tablet if you are able.

Creating an Effective Presence (Engineering)

I will be leading a section in this workshop on managing your digital footprint, developing and effective online presence, managing social media settings and options, as part of a wider session that looks at what it means to present yourself as a professional engineer and to evidence your skills and experience. 

This workshop on Tuesday 16th February (2-5pm), jointly hosted by the School of Engineering, the Careers Service and EDINA, will focus on Digital Footprint Awareness and creating an effective online presence to support summer internship and placement applications.

The session will include:

  • advice on using LinkedIn effectively;
  • an introduction to PebblePad for online portfolios;
  • guidance on managing your digital footprint.

Before attending, make sure you’ve registered for an account on LinkedIn. This is a BYOD session (bring your own device e.g. laptop or tablet).

Sign up (students in the School of Engineering only): http://www.innovativelearning.ed.ac.uk/creating-effective-online-presence-engineering

Communicating science to non-academic audiences ? who, what, why and how.

I have been involved in the planning of this session which I am contributing some social media, copyright/licensing and science communication expertise and resources to.

This science communication workshop explores how critical it is to identify your target audience and tailor your Open Educational Resource accordingly. The group will identify audiences and explore what their specific needs are before creating an interactive, web based, Open Educational Resource.

Sign up:

Other events worth noting include… 

The ILW newspaper (below) includes some highlights or you can search the programme in full here: http://www.innovativelearning.ed.ac.uk/ilw-calendar


And I’ll be sharing some of the resources from the sessions I’m involved with here on my blog (likely on the Publications and Presentations page).

Share/Bookmark

Social Media for Learning in Higher Education 2015 (#SocMedHE15) Conference – LiveBlog

Posted on by

Today I’m here at Sheffield Hallam University today for Social Media for Learning in Higher Education 2015 (follow #SocMedHE15) where myself and Louise Connelly (from UoE Royal (Dick) Veterinary School) will be presenting some of our Managing Your Digital Footprint research later today.

I’ll be liveblogging but, as the wifi is a little variable, there may be a slight delay in these posts. As usual, as this is a liveblog,

Welcome

At the moment we are being welcomed to the day by Sheffield Hallam’s Pro Vice Chancellor who is welcoming us to the day and highlighting that there are 55 papers from 38 HEIs. The hope is that today will generate new conversations and communities, and for those to keep going – and the University is planning to run the conference again next year.

Keynote by Eric Stoller

We are starting with a very heavily Star Wars themed video introducing Eric and his talk….

When he thinks about his day it has no clear pattern, and includes a lot of watching videos, exploring what others are doing… And I’m a big fan on Twitter polls (he polls the room – a fair few of us use them) and when you poll people about how universities are using social media we are seeing use for marketing and communications, teaching and learning, a whole range of activities…

There are such a range of channels out there… Snapchat, how many of you are Snapchatters? (fair few) and how many of you take screen shots? How about Reddit… yeah, there are a few of us, usually the nerdy folk… YikYak… I’m avoiding that to avoid Star Wars spoilers right now… Lots of sites out there…

And now what we say online matters. That is game changing… We have conversations in this auditorium and that doesn’t get shared beyond the room… But online our comments reaches out beyond this room… And that can be where we get into trouble around our digital identity. We can really thank Marc Prensky for really messing things up here with his Digital Natives idea… Dave White brilliantly responded to that, though few seemed to read it!

But there are some key issues here. Social media blurs professional and personal identities…

My dad was checking out Facebook but he’s not on Facebook, he was using my mothers account… My parents have given me a range of interesting examples of people blurring between different spaces… So my mom added me on Facebook.. Is she my friend? I think she has a different designation. I got on there and she already had 8 friends – how did they get there first? Anyway she is experiencing Facebook in a way that I haven’t for years… My mom joined Facebook in 2014 (“I wanted to make sure it wasn’t a fad”) and when you have 8 friends you truly see everything… She sees people that she doesn’t know making fun of, saying snarky things to, her child (me)… We’ve never really had a space where we have that blurring of people. So, my mom hops into a comment thread to defend me… And then people make fun of her… So I have to defend her… We haven’t really adapted and evolved our ways of being professional, of managing relationships for this space yet.

One thing we haven’t come to terms with is the idea of leadership in social media. No matter who you are you can educate, promote, etc. One of my favourite leaders on social media is in the US, president of the University of Cincinnati (@PrezOno). He has a lot of followers and engagement. Typically if your academics, your leaders, are using social media and sharing their work and insights, that says a lot about the organisational culture you are trying to build and encourage.

When you are thinking about employability (and man, you can’t miss this University’s employability office)… It’s about personal brand – what you post and say matters… It’s being human.

Facebook has been around 11 years now, it’s massive… There are over 1 billion users… In fact in September there were over 1 billion in a single day. But people don’t use it in the same ways they did previously… Look at institutions with an older cohort age then Facebook is where it’s at.

I have this quote from the University of Edinburgh’s Managing Your Digital Footprint account that 90% of bosses use Facebook to vet candidates… Which is potentially an issue… As students don’t always post that carefully or with an awareness of how their comments appear later on…

As a consultant I tell people not to fall in love with one platform, but I’m a little in love with Twitter. And there are really interesting things taking place there. We have things like #LTHEchat – a discussion of technology in education. And this is a space where comments are kind of preserved… But that can include silly comments, things we don’t want to stick around. And I love when universities connect students to alumni… We have to think about criticality and digital literacy in these spaces too…

Different spaces also work for different uses… Some love Vine, those 6 second videos. And when we think about teaching we want to talk about story telling some of the YouTube vloggers are a create place to learn about creating narrative and story. So, for instance, Casey Neilson, a vlogger who has also directed commercials for brands like Nike, is a great person to watch. For example his video on Haters and Losers… [we are now watching videos]

How many of you are on LinkedIn? [we mostly are] I assume those not on LinkedIn don’t have a job… There is huge amounts of useful stuff on there, including organisational pages… But it doesn’t always have a great reputation [shows a meme about adding you as a connection]. This is a space where we get our recommendations, our endorsements. Right now LinkedIn is a powerful place. LinkedIn is the only major social media site where there are more users ages 30-49 than 18-29 year olds [stat from Pew Research]. How many here work in employability or careers? You get that thing where students only approach you 5 minutes before they leave… They should really be getting on LinkedIn earlier. People can be weird about adding their students – it’s not about adding your students as friends, its an opportunity to recommend and support each other – much better there than Rate My Professor.

I wanted to show this tweet from the Association of Colleges that “soft skills should be called human skills. Soft makes it sound inferior, which we all know they’re not”. Those soft skills are part of what we do with social media…

When I moved to the UK – my wife got a promotion – and I, as a consultant, had all my networks in the US… But I also had social media contacts in the UK… And I was able to use LinkedIn groups, connections, etc. to build relationships in the UK, to find my way into the higher education sector here. I was talking to a LinkedIn rep last week at Princeton… What do you think the number one activity is on LinkedIn? It’s lurking… And I did a lot of strategic lurking…

So, we have these new spaces but we also have some older online spaces to remember…. So, for instance, what happens when you Google yourself? And that’s important to do… Part of what students are doing when they build up their profile online is to be searchable… To have great presence there.

And email still matters. How many of you love email? [one does] And how many of us have checked email today? [pretty much all]. We are all professional email checkers in a way… Email works if we do it right… But we don’t. We send huge long messages, we reply all to unsubscribe… It’s not surprising if students don’t get that [cue a tweet that shows an email tactically bearing a subject line about free football tix miraculously was received by students].

How many of you are concerned about privacy on social media? It’s always a huge concern. We have spaces like Snapchat – ephemeral except some of you take screen shots – and Yik Yak. We’ve already had issues with Yik Yak – a lecturer walked out when she saw horrible things people were posting about here… But Yik Yak tends to be sex and drugs and Netflix… Also a lot of revision…

And we have Periscope. Twitter owns it now, so who knows where that will go… It’s a powerful tool to have… You can livestream video from anywhere, which used to be hugely difficult and expensive. And you get comments and discussion.

And you don’t need to always do social media by posting, there is so much to listen and learn from…

The student experience is holistic. Social media, just like it blurs personal and professional selves, the same thing happens with teaching and learning and higher education. There are not separate entities in an organisation now… academic advising, careers services, induction/orientation, first year success, mental health/wellness…. So much learning happens in this space, and it’s not necessarily formal…

There is no such thing as a digital native… there are people learning and trying things…

So, now, some Q&A.

Q&A

Q1) When you see lecturers named on YikYak… Can you really just ignore it?

A1) On YikYak the community can downvote unpleasant bad things. In the US a threat can be prosecuted [also in the UK, where hate speech laws also apply]. But if I say something insulting it’s not necessarily illegal… It’s just nasty… You get seasonal trolling – exam time, venting… But we have to crack the nut about why people are doing and saying this stuff… It’s not new, the app just lets us see it. So you can downvote. You can comment (positively). We saw that with Twitter, and we still see that on Twitter. People writing on pointed issues still get a lot of abuse… Hate speech, bullying, it’s not new… it’s bigger than social media… It’s just reflected by social media.

Q2) On the conference hashtag people are concerned about going into the open spaces… and particularly the ads in these spaces…

A2) I am a big fan of adblock in Chrome. But until this stuff becomes a public utility, we have to use the tools that have scale and work the best. There are tools that try to be Facebook and Twitter without the ads… It’s like telling people to leave a party and go to an empty room… But if you use Google you are being sold… I have so much commercial branded stuff around me. When our communications are being sold… That gets messy… Instagram a while back wanted to own all the photos shared but there was a revolt from photographers and they had to go back on that… The community changed that. And you have to block those who do try to use you or take advantage (e.g. generating an ad that says Eric likes University of Pheonix, you should too… ).

Q3) I find social media makes me anxious, there are so many issues and concerns here…

A3) I think we are in a world where we need discipline about not checking our phone in the middle of the night… Don’t let these things run your life… If anything causes you anxiety you have to manage that, you have to address that… You all are tweeting, my phone will have notifications… I’ll check it later… That’s fine… I don’t have to reply to everyone…

Q4) You talked about how we are all professional emailers… To what extent is social media also part of everybody’s job now? And how do we build social media in?

A4) In higher ed we see digital champions in organisations… Even if not stated. Email is assumed in our job descriptions… I think social media is starting to weave in in the same ways… We are still feeling out how social media fits into the fabric of our day… The learning curve at the beginning can feel steep if everything is new to you… Twitter took me a year or two to embed in my day, but I’ve found it effective, efficient, and now it’s an essential part of my day. But it’s nice when communication and engagement is part of a job description, it frees people to do that with their day, and ties it to their review process etc.

Workshops 1: Transforming learning by understanding how students use social media as a different space – Andrew Middleton, Head of Academic Practice and Learning Innovation, LEAD, Sheffield Hallam University

I’m assuming that, having come to a conference on social media in learning, you are passionate about learning and teaching… And I think we have to go back to first principles…

Claudia Megele (2015) has, I think, got it spot on about pedagoguey. We are experiencing “a paradigm shift that requires a comprehensive rethink and reconceptualisation of higher education in a rapidly changing socio-technological context where the definition straddles formal and informal behaviours” [check that phrasing].

When we think about formal, that tends to mean spaces like we are in at the moment. Michael Errow makes the point that non-formal is different, something other than the formal teaching and learning space. In a way one way to look at this is to think about disruption, and disrupting the formal. Because of the media and technologies we use, we are disrupting the formal… In that keynote everyone was in what Eric called the “praying” position – all on our phones and laptops… We have changed in these formal spaces… Through our habits and behaviours we are changing our idea of formal, creating our own (parallel) informal space. What does that mean for us as teachers… We have to engage in this non-formal space. From provided to self-constructed, from isolated to connected learning, from directed to self-determined, from construction to co-construction, from impersonal to social, and from the abstract and theoretical to authentic and practical (our employers brief our students through YouTube, through tweet chats – eg a student oncology tweet chat)

 

11:20-11:35 – Refreshment Break

11:35-12:05 – Short Papers 1

12:10-12:40 – Short Papers 2

12:40-13:40 – Lunch

13:40-14:40 – Workshops 2 (afternoon) 

14:40-14:55 – Refreshment Break

14:55-15:25 – Short Papers 3

15:30-16:00 – Short Papers 4

16:00 – Conference ends

Share/Bookmark