How can we assess the writing of our students in ways that are valid, reliable, and fair? That is the subject of the latest podcast episode of Teaching Writing:
Writing assessment: An interview with Dr. David Slomp
In this 30-minute conversation with Dr. David Slomp, Associate Professor of Education at the University of Lethbridge and co-editor in chief of the journal, Assessing Writing, you'll find out how to create assessments that satisfy all three of these criteria. "Valid" speaks to the point that your assessment tool must really assess the characteristic you are measuring. "Reliable" means several things, including that the test or assessment tool gives the same result. And "fair" asks us to consider if all the people who are subject to the assessment have an equal opportunity to perform the task or skill being assessed.
If you a manager and are assessing the writing of people in your workgroup, these three cornerstones apply equally to both summative assessments (year-end reviews) and formative assessments--the kind of coaching or feedback you give on a daily, weekly, or monthly basis. Do some people in your group receive more difficult assignments? Less time to work on them? Fewer comments or opportunities to revise?
Good assessments are difficult but extremely useful if they give you a good picture of the overall effectiveness of your work group and/or a clear sense of progress or lack of it for those in the group. If some people aren't improving, and you have good data about that, you can then work with them to find ways to get them help with their writing: coaches, seminars (online and in-person), and even peer mentoring. For more information about some of the resources out there, visit my website and check out the online courses available through LinkedIn's Learning page.
Audience, or perhaps "readers" is a better term for text/discourse, defines many communication situations. That is, who you are talking to or writing to is often the most important aspect of a document or speech. The problems for writers are at least twofold: helping them understand this component of a communication, and helping them figure out how to adjust what they plan to say to that audience.
Audience is really part of a larger concept called a "rhetorical situation." Lloyd Bitzer coined this term to refer to the way that language (that is, rhetoric) can be used in situations in order to move towards a solution to the problem under discussion. When we write or speak in a situation we always do so for an audience (and almost always an audience external to ourselves). For this reason, we need to understand that audience so that we may adjust our discourse to them. These adjustments of language are the "rhetoric" of the situation.
In my most recent podcast, "Audience and rhetorical situations," I talk about how audienceis a key component of rhetorical situations; exigence (the perhaps best thought of as the problem or issue under discussion) is another key component, along with constraints (the range of possible solutions). These components are important both for academic writing--the exigence for my podcast--but also for workplace writing. For LinkedIn readers, workplace exigencies abound: what demographics should we focus our advertising on? Which project is most important to our unit? Where are we in our progress on our strategic plans?
Even more importantly, workplace communication tends to target audiences in much more specific and obvious ways than academic communication. For this reason, when you write or communicate at work you need to have an even more sophisticated understanding of why your are writing (the exigence), who you are writing for (are you writing to someone more or less powerful), and what the possibilities are for solving a problem through language (the constraints).
So what should you have remembered or taken from the writing courses that you took as a post-secondary student? What counts as good writing is highly context specific. That is, good writing in school contexts is not what counts as good writing in workplace contexts. More specifically, many workplace contexts differ from one another in what they value as good communication. Your job is to have a good theoretical understanding of communication--exigence, audience, constraints--and apply that understanding to the workplace you find yourself in.
For more information about this and other podcasts, visit wecanwrite.ca, follow me on Twitter (@rogergraves), or visit the Writing Across the Curriculum page at the University of Alberta.
How does an overall sense of the rhetorical situation help a writer move forward and write well? In earlier episodes of my podcast, Teaching Writing, I considered how audience and purpose contribute to an answer to that question. Today I want to complicate and add to those two factors by considering the genre that a writer--at work or in an academic setting--faces.
So, to refresh: when you are writing, you need to know why you are writing--what is the point or purpose? At work, this is usually pretty clear cut. You email to follow-up with someone on a project, confirm a course of action, request information, arrange a meeting, and so on. Some documents, however, have more than one purpose, and that can complicate things considerably. The second main variable to consider is audience: who will read your message? With email particularly, there is a "To:" field that answers that question; keep in mind, however, that email is easily forwarded and the initial or primary audience you have in mind may ultimately be only one of the audiences that reviews your message.
For more extended texts, such as reports of various kinds or performance reviews, researchers have noticed that the genre or kind of document itself exerts a great influence on what can be written. The "work" a writer wants to do by creating the document--get a raise, pass a probationary period--is constrained by the genre. You need to write within the limits of what a specific kind of document allows. When writing at work, your organization probably has unwritten expectations for the documents produced within it. A year-end review is probably not the right place to pitch an idea for a new marketing program.
In some cases, lower ranking employees may be asked to write background reports--white papers, for example--that provide information about a topic or new product without recommending a course of action. The person requesting that report will use it in discussions with those further up the hierarchy to come up with an action plan. Knowing your place within the group is essential information for deciding how much of a recommendation or argument you should include in your documents.
If you are supervising new employees or people new to your workgroup, consider sketching out a plan for them to learn the written genres that are important to your work. Partner them with a more experienced employee who can review their documents before you see them; create a file or folder with examples of good work that they could review; perhaps make point form notes about what you are looking for in the documents you request.
These are all techniques for helping people learn new genres. Try to keep in mind that the way things are written in your organization may vary in important ways from other, similar organizations. Don't assume that new employees will bring specific knowledge of how to write well for you, and do put a plan in place to help speed their learning and acculturation into your organization.
Roger Graves, wecanwrite.ca
There is no denying the ascension of the visual in communications of all sorts today. Take a look at any random sample of non-technical documents and you will see evidence everywhere. Even in technical and business communication, visuals claim a prominent if not central role. In a recent podcast episode of Teaching Writing, I examine the range and function of visual information in documents.
What does that mean for you as a writer? First, you need to notice the visual elements in the document types you create. How much of a visual component is there? What kinds of visuals are typically inserted? It is important to remember that the visual component includes photos, graphs, graphics, illustrations, tables, maps, and even headings. Online documents--or documents primarily shared digitally--no longer cost more to create or print, so writers typically have a wide range of colours available to them. So in addition to the obvious visual component writers need to think about which colours to use for text, lines, backgrounds, and headings.
An efficient way to deal with all of these choices and requirements is to use templates, either ones associated with the software you are using or ones prepared by your organization. These templates often have colour choices and typefaces pre-selected for you. Many word processing applications provide templates for typical documents that you can work from. Use them.
But in your writing at work, notice the kinds of visual information that your documents typically include. Beyond that, focus on the work or the function of those visuals. Why are they there? Are they primarily decorative? Do they contribute to branding? Do they provide evidence or information that duplicates what is said in the text? Do they elaborate on what the text is able to say? Do they reinforce the text and make the point more vividly?
Once you have a sense of what the function of each visual is, you will have a greater understanding of the kinds of visual information readers expect to see in your documents. You might also see opportunities for including more visual information, and that, in turn, might make your writing more effective.
There is one more potential spinoff: if you develop some skill at translating textual information into visual information, you will be able to create much more interesting slides for your presentation.
For more information on how to incorporate visual information in documents, check out my technical communication textbook, a Strategic Guide to Technical Communication, and visit my website at wecanwrite.ca
At the start of a recent workshop, I asked graduate student supervisors what they most wanted to know. While the answers are of interest to supervisors, graduate students might gain some insight from listening in. Research labs function as small development units, so there may be value in listening to the answers if you are a supervisor of a non-academic unit at a large organization.
In this episode of my Teaching Writing podcast, I provide an edited version of my answers. Here are the questions:
Q1: Efficiency. How can you get your co-workers writing better, faster. I've had success in the past working one-on-one with people who, at the start of our relationship, were having trouble producing texts that their supervisors liked. We had success by studying models of good writing drawn (often) from the supervisor's written work.
2. Feedback. Providing good feedback takes time and is not particularly efficient. More importantly, research suggests that the person providing the feedback is the one who is doing most of the learning. So if you are providing great feedback to your employees, you are learning a lot about their writing but they aren't learning nearly as much as you are.
Solution: structure opportunities for co-workers to comment on each other's work. Reward those who provide constructive, insightful suggestions.
3. Non-native speakers. Non-native speakers vary widely in their ability to write in English. Some are fantastic and indistinguishable from native speakers. Others will require a lot of help.
For sentence-level help, I would take the second group--those who need help--and insist that they use a grammar editor before submitting documents for review. Google docs have several add-ons, and programs such as Grammarly can be structured to work with browsers as well as various word processors. This will help those writers produce standard English, but it won't help with the larger concerns about how to structure a document appropriately. For that, you'll need to do what has been covered in the first question above.
More than anything, you need to start with a new student or employee by getting a sample of their writing and identifying what they need to work on to improve. Then you need to put together a plan for helping them improve. Maybe that means $35 for a grammar checker; maybe that means having co-workers review their writing; maybe that means creating models of good writing with notes to explain why they are good. But the key is having a plan.
To find out more, visit wecanwrite.ca or email me at firstname.lastname@example.org for advise or suggestions.
Recent research into online peer feedback shows that people who give feedback to others on what that person has written benefits the person giving the feedback more than the person receiving it. This isn't typically what people expect: they will pay for feedback on their writing and then revise it. That will solve the immediate problem with that specific piece of writing, and if that solves your current problem, great!
To improve as a writer in the long term, however, you need to go beyond receiving feedback. You need to learn to "read as a writer": find texts similar to the one you are trying to create, and then analyze them to find out how they are put together. If you then try to explain (to yourself, if you do not know the author of the text you are analyzing) to the person who wrote the text how you see the text working (or not working!), then you've taken the next step.
Over time, you ability to see opportunities for revising your own work will improve. It helps if there is some standard or criteria for your text. We can help do an initial analysis to identify the features of your target texts, and we can also help you improve your ability to comment on other writer's texts in ways that are productive for you and them.