Category Archives: Online Teaching and Learning

Moodle Assignment Feature: Grading options (rubrics, marking guides)

This third post in a series of Ed Tech Blog posts focusing on different options within the Moodle Assignment Activity focuses on different types of grading functions.  So far this semester, I’ve introduced the options of the Download All, Offline Grading Sheet, and the batch upload features of this tool to help grading be a bit less time consuming (less downloading/uploading time).  Today I describe two “Advanced” grading features available in the Assignment Activity: Rubrics and Marking Guides.

 

Many of us use rubrics when creating our grading schemes especially for larger papers and projects.  Rubrics like this are very helpful for both instructors and students because they more deliberately outline the expectations for student work.  Research does demonstrate that the simple act of providing (and describing) rubrics prior to students beginning work on their projects can help to increase student performance (cite).  Of course the students need to actually pay attention and use these rubrics as they build and revise these projects, but it does help facilitate communication of expectations.

 

You can build your rubrics right into Moodle assignments in two ways: Rubrics (which creates boxes that you select) and Marking Guides (allows for ranges of scores to be placed in each criteria).  Here is a visual example of each type of advanced grading:

 

Moodle Rubrics:

 

Each level of each criteria has a description of the expectations and a set point value.  Instructors designing the rubric can determine the point value for each level. Rubrics make for quick grading because graders then simply click the level achieved by the student.  However, there is no option to alter the point values for each ‘cell.’

 

 

GIF of a Moodle Rubric

Moodle Marking Guides:

 

Criteria are listed and described and a total point value is determined for each criteria.  Graders are able to type in feedback into textboxes for individual criteria and for overall comments.  Graders are able to select a point value range that is allowed by each criteria.

 

 

The benefits of building in these types of grading structures within Moodle include being a bit more efficient with grading and scoring.  Moodle will automatically calculate the total score for you when you use these features. Rubrics are nice because graders can just click the cell they determine the student work to be at.  Additionally, you have the option of making these grading criteria for marking guides and rubrics visible to students within the assignment activity itself, so students are aware of how they will be evaluated.  Additionally, when students receive a grade they see the same rubric or marking guide and can see why they have missed points or done well.

 

Some weaknesses of using these grading structures include not being able to use the Offline Grading Sheet to batch upload grades.  You can still to a batch upload of feedback files, but because of the need to enter in the individual rubric criteria, the offline grading sheet is not possible.  Another weakness specifically with the Rubric option is that the points allocated to each cell cannot be altered when grading. What I mean by this is that clicking a rubric cell is an all-or-none action.  Keeping this in mind when building the rubric is important. In my own use, I have found that I sometimes ended up giving much higher or lower scores than I believed was really warranted because I was tied to the points on the rubric and these did not allow me to give more nuanced grades.  I personally tend to find the Marking Guide a nice compromise to this issue because I am able to give a range of points within a grading criteria.

 

Building these can be a bit time consuming and I would recommend including these in your courses after you have built up most of the rest of your course first.  You can always grade using the simple grading method. It is also possible to create rubrics that you use for multiple assignments (for example, the same rubric is used for all drafts of a paper or is used for all of a particular type of assignment).  Below is a series of videos on how to create Rubrics and Marking Guides in Moodle.

Creating a Rubric in Moodle    [Scroll down to see how to Create a Marking Guide]

Creating a Marking Guide in Moodle

Morningside College QM Plan. What is it? How do I learn more?

Many of you are likely aware that Morningside College subscribed to Quality Matters, but did you also know that we have an active plan to help us implement the Quality Matters standards into our online courses? One of the services that Quality Matters provides is a process to help institutions create implementation plans.  A college representative (typically the campus Quality Matters Coordinator or QMC) completes a proposal that follows the guidelines given by Quality Matters to describe the process that the institution will take to help implement the Quality Matters standards within is online and blended courses.

 

A couple of years ago, I stumbled across this service and decided to create a QM Implementation Plan for Morningside College which focuses on creating a form of internal review process for our online courses in our online programs (Graduate Education, Graduate Nursing, and Organizational Management).  I worked with John Pino, Steve Gates, Jackie Barber, and Michelle Laughlin as I drafted this proposal. What I liked about the process was QMs flexibility for us to create a plan that fit Morningside and our needs. Our needs were to create a review system for ourselves, guided by the QM standards. I submitted this proposal to QM and they basically said “Great! Go for it!” You can read the specifics of this plan here if you are interested.

 

We are currently in Year 2 of a three year plan.  Last year I recruited several faculty and adjuncts within the Graduate Education program to complete the training necessary to become peer-reviewers for our internal process.  This year, I hope to continue to recruit interested faculty and to begin training those who have completed the training to conduct reviews. Ultimately it is my goal to try to have at least 2 courses from each program reviewed internally.

 

So what does it mean that we have a QM ‘approved’ implementation plan? Each year I submit an annual report indicating our progress on our stated goals.  And we are also able to include a statement in our web pages that states:

Morningside College is committed to implement the Quality Matters Standards for the design of online and/or hybrid courses, and we are systematically building and evaluating our courses based on these rigorous, research-supported standards. The Quality Matters standards assure that the online components of these courses promote learner engagement and provide students with all the tools and information they need to be successful learners. More information regarding Quality Matters may be found at https://www.qualitymatters.org/.

 

Beyond the request for annual reports there is no other ‘policing’ that QM does.  So it’s not like we will be punished for not meeting our goals. But they do provide guidance and resources for those of us trying to accomplish our goals.

 

If you are interested in learning more about the Morningside College QM Implementation Plan, please feel free to contact me.  If you are interested in becoming an internal reviewer I would love to talk to you more! Even if you do not teach online, you can become a reviewer.  Becoming a reviewer for our online programs will allow you to see just what our online courses look like, how our online students are learning, and it can even give you ideas for your own courses!

A Primer on Quality Matters

During the 1/19 faculty meeting I described a little bit about the organization Quality Matters. Quality Matters is an organization that grew out of a FIPSE grant from MarylandOnline in an effort to improve and provide standards for online and blended course delivery. After the three year grant came to a close, Quality Matters became its own entity and is now funded by subscriptions. To date there are over 900 institutional subscribers.

Quality Matters provides tools and services to educational institutions to assist in the improvement of online and blended learning. Some of the tools that are provided are the rubrics based on the Eight Standards identified by QM as important for online learning, training courses for developing online and blended courses, using the QM rubrics, and becoming an official QM peer-reviewer and they also provide the service of coordinating and conducting individual course reviews. Benefits of being a subscriber is having access to these tools at a reduced price and being provided with a structure to manage internal course reviews within the institution.

Morningside is currently testing out the utility of the Quality Matters rubrics and exploring the benefits of continuing a subscription to their services.  Grad Nursing has been using the rubrics in their course development for this semester and plans to conduct some peer reviews of these courses later on. Grad Ed is planning on applying the QM rubrics to courses within their Foundations Core to assist in course improvement. A few faculty teaching online undergraduate courses will participate in a pilot using the QM rubrics on their courses.

Additionally, I plan to complete several (three) of the online professional development courses offered by Quality Matters (Applying the QM Rubrics, Designing Online Courses, and Designing Blended Courses) over the course of this semester not only to improve my own understanding in these areas, but to evaluate the quality of these professional development opportunities.

Later on in the semester (April 9) FDC will hold a workshop on Online Course Development where myself and several others who have been involved in using the QM rubrics will describe our experiences and methods of using these tools.

As we move toward the creation of new programs that are online and in increasing online offerings for our residential students, the issue of quality is paramount. Quality Matters provides a national (really international) benchmark for quality of online teaching. Whether we stay with Quality Matters will depend on the value that we see from these pilots. But if it is not Quality Matters, it will be something else similar that will be used.

 

 

Pi – A different venue for electronic communication with students

Earlier this summer I was contacted by a gentleman of a new Internet company that is developing a new communication application for the classroom (online or traditional). The product is called Pi (https://piapp.co/) and it basically uses a Twitter/Facebook-type interface and brings it into a course LMS (instructor just links to the Pi course page from the LMS).

The creators of this application voice their frustration with the traditional communication methods used in learning management systems (i.e. e-mail and forums) finding them to be less engaging with students not really interacting with one another as much as they could. So they set out to create an improved method.

The major differences as I see it are two-fold. 1) the interface is more like a social media platform that many students are very familiar with and 2) there are functions that push students to be more actively involved in discussions within this interface.

This is what the computer version of Pi looks like:

Pi Screenshot

As you can see it looks a little like a twitter feed, but has more of the posting ability of Facebook (not limited to 140 characters). This is a familiar interface for students, more so than some forum interfaces.

There are also mobile versions of this for both iOS and Android (not sure about Windows). Students can have the option of having push notifications active for Pi on their phones, which means, when there is a new post or reply they get an alert. Rather than depending on students going to visit a forum site to see if there are new posts or replies, now their device can alert them immediately. There is also the option of subscribing to e-mail alerts which again directs users to the conversations.

If you are interested in learning more about Pi, feel free to contact Brent Burd for a free account (Brent@piapp.co). Just let him know you are an instructor at Morningside College and are interested in testing out the product.

If you want to join my test class in Pi you can use this link (http://piapp.co/join/c439c) to join up and become a member of this ‘course.’ We can use this as something of a sandbox to play around with if you please.

For more information here are the online sites for Pi:

Website: https://piapp.co/

Twitter: https://twitter.com/piappHQ

Facebook: https://www.facebook.com/piapphq?fref=nf

 

The Importance of Research in Instruction

For the past few months I’ve been reading Clark and Mayer’s “E-Learning and the Science of Instruction” in an effort to gather some of the most up-to-date research in online learning. These authors use psychology and cognitive theory to make recommendations on how to best deliver online learning. They use the most recent research to support their recommendations. Because these are based solidly on cognitive research (my own research background) I am strongly drawn to these practices.

My own background is in cognitive psychology, however, this does not mean that I was not a little surprised by some of the research findings and recommendations. In their chapter “Applying the Redundancy Principle” (Snazzy name right!), they make a recommendation that frankly surprised me at first. When describing a visual of central importance, DO NOT include both audio and redundant text at the same time. This went against what I had been taught in terms of making learning accessible to individuals with disabilities. One of the things that I recall when I was in graduate school was a push to make all learning materials accessible to as many abilities as possible. During that time our department was spending a lot of time and resources close-captioning old video tapes and DVDs. I did a grant proposal to create an online learning module using something called Universal Design during this time as well. The idea of Universal Design, if you have not heard this term before, is really simply the idea that when something is designed, it should be useable by most (if not all) people, and that the accommodations provided in the design would benefit not only those with some form of disability or limitation, but would benefit all.

Take this example (close to my heart as a mother with two small children). Designing buildings with automatic doors allows not only those who may have mobility limitations, but also come in really handy when carrying a toddler, a purse, a diaper bag, and a sack of groceries. Designing sidewalks with slopes to the street allow wheelchairs to navigate that pavements and allows for  smoother riding for bicyclists, skaters, and strollers.

This same principle was really being encouraged in the learning environment when I was in grad school in the early 2000s. The idea that providing things like audio and text together for computer lessons would allow not only those with hearing or visual handicaps to access the information, but could also appeal to students with different preferred ‘learning styles’ (I’ll post a subsequent blog on my thoughts about learning styles in the future). In other words it’s a win-win situation and the added information does not affect the overall learning of the material. Thus, it was often recommended that both the audio and text be made easily available and this was the opinion that I held until I read this chapter in Clark and Mayer the other day.

Clark and Mayer actually suggest that if the information is redundant (i.e. the text is simply a repeat of the audio) and the audio is being used to explain or illustrate some visual image (say an animation or simulation), then the text should NOT be made visible on the screen because doing so would actually hinder learning. Frankly this surprised me and left me questioning, but what about the idea of universal design and accessibility for learners with hearing impairments? The research they described explained their reasoning and they also made a recommendation for the accessibility issue as well.

In cognitive science the prevailing theory is that our cognitive system is made up of different components that process different forms of information. To explain why you should avoid having redundant text and audio when describing a visual I will focus on two: the articulatory (or phonological) loop and the visuo-spatial sketchpad.

The articulatory loop is the center for processing auditory information. It is primarily ‘verbal’ in nature and is best described as the ‘voice in your head’ you use when processing something. The visuo-spatial sketchpad is the center for processing visual and spatial information and is best described as your ‘mind’s eye.’ These two processing centers do not interfere with one another, but processing multiple pieces of information within one of these does cause interference.

Here is an example of interference. If I were to say to you to count the number of words in the following sentence: and I then said “How many words are in this sentence,” odds are to count the words you would ‘say’ the sentence again in your head, but possibly use your fingers to maintain a count. Most people do this because they are unable to both think the sentence and keep a mental count at the same time. These are two types of ‘auditory’ pieces of information and the articulatory loop basically gets overloaded and cannot process this well. Therefore you use your fingers to keep count so you are only processing one piece of information.

On the other hand to illustrate how the articulatory loop and visuo-spatial sketchpad do not interfere with one another try this exercise: Think of the home that you spent most of your childhood in. Now tell me how many windows were in that house. Odds are here that you do not need to use your fingers to count. You will use your mind’s eye to virtually walk through the house and you will keep a mental count. The walk through the house uses the visuo-spatial sketchpad, the mental count uses the articulatory loop. You are able to do both mentally because there is not the interference like with the previous example.

So this is the theory behind why both audio and text should not redundantly accompany some visual of central importance. Having the visual (e.g. an animation, simulation) uses the visuo-spatial sketchpad and the audio uses the articulatory loop to be processed. If you include also visual text, this will also use the articulatory loop (yes, it’s visual, but primarily processed in an auditory manner). This means that this may interfere with the learner’s ability to keep up with the instructions that are accompanying the visual (the audio and text begin interfering with one another reducing the speed and accuracy of processing). Thus, these authors suggest not including the text for this reason. Additionally, having the instruction in an auditory manner allow the learner’s eyes to focus only on the visual and not jump back-and-forth between text and visual, another bonus for learning.

So what about those with hearing impairments? Clark and Mayer recommend having an “Audio Off” option button, which would turn off the audio and then only provide the text form of the instruction. They recommend that this function NOT also include the audio.

The title of this blog entry is “The Importance of Research In Instruction” and I believe that this example is perfect for demonstrating why it is so important not only to base instructional decisions on research, but to keep current in the research. The recommendations for providing both audio and text have changed. That is the nature of science. We keep investigating and learning more. Sometimes, what we once believed to be true is discredited by better design or different conditions. I hope to assist in providing the most recent recommendations based on the literature to you all.

If you are interested in more about online learning or even just want to learn more about what cognitive science tells us about how we best learn, I highly recommend this book. What I like most about it is not only its empirically based recommendations, but the fact that these authors also describe the “boundary conditions” for their recommendations. These are the exceptions to the rules and again are based in empirical evidence. For example, in the principle being described in this blog, you may choose to include SOME text such as key words in a process or some technical vocabulary on the screen in addition to the audio, but doing so should be under only certain conditions. Using a one-size-fits-all approach in all cases is not ideal, but when we do deviate from the recommendations, we should do so purposefully and in a way that does not hinder learning.

Tips for facilitating online discussions

One way that online instructors can greatly increase their presence in an online course is by utilizing forums for discussion. This provides an opportunity for all three forms of communication (student-to-resource, student-to-student, instructor-to-student). Instructors can interact in a variety of ways and can help foster discussion and learning by using different types of reply posts. It is important to note, that instructors should not feel obligated to respond to each and every student post. Doing so would be time consuming and we would not really expect this in a face to face conversation either. However, being clear about this to your students at the beginning may be helpful (i.e. state that you will be responding to posts, but that responding to each individual student post is not likely/reasonable).

To help instructors increase their presence and to help better facilitate learning and class discussion, the following tips can be used.

  1. Connect a comment or post from a student to previously learned material. In student learning, making explicit connections between old and new material helps to build a stronger network of knowledge. Students will sometimes do this themselves spontaneously, but other times they need help.
    • Example instructor posts:
      • Your comment reminds me of the concept of X that we discussed two weeks ago. Can you explain how your thoughts and concept X are related?
      • Very interesting example from your personal experience. Can you tie this back to a theory that was discussed earlier in the class relating to this example?
  2. Ask for elaboration or clarification. Perhaps based on a rubric guidelines for forum posts. Asking for further elaboration requires a student to basically write-out their thought process. This activity of outlining their thoughts helps both the instructor and the student see why they responded the way they did and helps to increase metacognitive awareness.
    • Example instructor posts:
      • Why do you think that?
      • What is your reasoning?
      • Is there an alternative strategy?
  3. Prompt another student to respond. Don’t feel that you, the instructor, needs to personally respond all the time. Prompt others to chime in. Students have valid thoughts and experiences that can be meaningful to the conversation.
    • Example instructor posts:
      • Student X gives an excellent example in this situation. Can others give an example of this in other contexts?
      • Student Y believes ______. What do others believe?
  4. Give a question to a student to research and answer. Perhaps your student is more of an expert in an area than you are (yeah, it happens). Ask a question that you have and ask the student to respond back to you. Of course the student does not need to be an expert, but they can get practice in researching a question and replying back.
    • Example instructor posts:
      • You mention using Method A with your classroom. I’m not familiar with this approach; can you explain this more to me?
      • You describe a disorder that appears to be quite rare. Please do a little additional research about what is known about this disorder and how it might be treated.
  5. Ask other students to respond to a student’s question. Too often students see their instructor as the only source for information, when in reality the students themselves can also be resources to one another. To help facilitate students serving as informational resources to one another try to give the responsibility for answering questions to them.
    • Example instructor posts:
      • Student A asks an interesting question about Concept Z. I want another student to respond and answer to her question.
      • I could answer your question directly, but I think Student B had a good description in his post earlier. Student B can you address Student C’s question?
  6. Make specific comments on particularly well thought-outposts. Especially important early on in the course, pointing out well-done posts and why they are well done can help students model future posts. When doing this, identify specifics about why the post was particularly good. For example, did the student elaborate and explain his or her thoughts, did the student explicitly connect their experiences with the course content, did the student reference back to one of the course materials or scholarly materials in the field?
    • Example instructor posts
      • Student K, this post is extremely thoughtful. You explain how Theory A ties into the practices that you use within your own job and why Theory A tends to be more effective than Theory B in your situation. You also refer to the research article comparing Theories A and B which explain why these two approaches are present.
      • Student Z, you have clearly tied your own experiences back to the concept of Y and use the readings from this unit to support these connections. Additionally, you mention how the goals of this course will help you to improve your understanding of concept Y and its use within the field.
  7. Reinforce the use of newly learned course material: This ties back to Tip #1 but instead of the instructor making a comment about how new material connects to previously learned material, the student has done so him or herself spontaneously. When this occurs, be sure to reinforce making these connections in some way so that this student and other reading the posts continue to consider how new information relates to old.
    • Example Instructor posts:
      • Good job connecting how concepts X, Y, and Z all connect back to the earlier idea of A. These ideas are all connected.
      • You note how the earlier definition of concept A is important to understanding Theory B in this unit. Good job at seeing this connection.
  8. Validate experience: Though using only personal experience and anecdotal evidence is likely not appropriate, it is important to validate a person’s personal experiences. You can nudge students to go beyond simply giving personal experiences by using Tips #2 and #6. Ask them to use the context of their experiences to evaluate some information from the course.
    • Example instructor posts:
      • I understand that you have had a difficult time with your students when using Method A. It can be frustrating when what is considered a tried and true method doesn’t seem to work. Do you think there is a weakness within Method A that might be contributing to your frustrations and if so what about Method A might need adjustment?
      • It does seem like common sense would suggest that doing _____ would work better and you give a few examples of when this is the case. Can you think of situations in your experience when using the theory discussed in class might also work?
  9. When the forum ends, post an overall summary of major points raised, issues discussed, resolutions to issues, and continued points of disagreement. A summary will help tie what could be a disjointed set of conversations into one. To do a good summary instructors will need to monitor discussions boards often and take notes about how the conversation(s) progress. Tie items back to the class materials and outcomes when possible. Take note about continued disagreements within the class and how these might mirror continued disagreements within the field.
    • Example Instructor posts:
      • This forum was focused on Topic A by asking students to respond to this question “……….”. It appears that many people believed that Point 1, Point 2, and Point 3 were most relevant to their jobs, but Point 4 was often not present or was very rare. This is also what the research suggested from Reading 9. Most agreed that Method A seemed to work best for them. However in the situation of Q there appeared to be continued debate about whether Method B or Method C were most effective. This debate is also present in the research as you can see from Reading 10.

 

These tips were suggested by and/or inspired by content from the following Best Practices resources:

  1. http://www.designingforlearning.info/services/writing/ecoach/tenbest.html
  2. http://facdev.e-education.psu.edu/teach/bestpractices
  3. http://www.brown.edu/academics/professional/faculty/online/best-practices.php