DRAFT: This module has unpublished changes.

 

Global And Cultural Awareness AIM Team

 

2019-2020 Scoring Year

 

Spring Semester

 

Results from scoring shared at results site.

 

April 3, 2020 - scoring folder shared with scorers - return date is Friday, April 25th.

 

April 1, 2020 - 12 noon - virtual norming session via Zoom (recording here)

Members present: J. Alonso, M. McCamish, T. Williams, A. Wood, R. Aslinger, N. Petty, A. Sanders, M. Loisy. Members also wanting to score but not present: N. Mansito, A. Kaufmann, K. Thomspon.

The team discussed 2 artifacts and determined fair performance scores. 2 additional artifacts were shared with scores for reference during scoring window.  Each scorer's packet will have these normed resources available.  Each scorer will have no more than 15 artifacts to score and packets will be available Friday, April 3.

 

Fall Semester

 

 

DRAFT: This module has unpublished changes.

Global and Cultural Awareness Team Updates

 

2018-19 Scoring Year Activities (reverse chronological order)

 

4/17/2019

Meeting with Team Leads and IERP for review and planning for next steps.  Slight adjustments made to summary report which will be posted soon.

Plenary session will be devoted to feedback from evaluators and constributing instructors (will send personal invite).  Early fall will be devoted to slight adjustments to areas of the rubric where areas of concern were raised during norming: Skills (Communication) regarding whether the paper itself is the communication and Cultural Diversity regarding the definition of subcultures.  

Plan to collect artifacts from fall 2019 and score again in spring 2020.

 

4/8/2019 - 4/15/2019

Summary Report emailed to Team Leads for review.  Individual Instructor Reports emailed to appropriate constributing instructors.

 

3/1/2019 - 3/22/2019

Scoring window opened/closed

 

3/1/2019

Norming session - 10 am 

Members present: J. Alonso, M. McCamish, A. Sanders, M. Spear, A. Kaufman, R. Aslinger, A. Wood, M. McCarthy, N. Mansito, T. Williams

An aggregated score sheet for each artifact was shared with team members.  Discussion began with ANTH1230 and ENGL1010_norm2 artifacts because 1) between the 2 artifacts, all areas of the rubric were evaluated and 2) there was disagreement among team members on student performance levels for each artifact.  

Team members were reminded that evaluation using the rubric is not the same as grading the assignment and that in order for performance to land in a particular performance level, all descriptors of the performance level must be evident.  There are no between scores such as 1.5 or 2.5 - these scores round down to the highest level at which all descriptors of performance are evident.

Members discussed specific examples from each artifact and agreed on a common viewpoint for evaluating those areas of greatest disagreement. The final normed scores were agreed and will serve as a resource for evaluators during the scoring window (along with the normed artifacts)

 

After meeting:

Email went to scoring team members (12) with access to normed artifacts and resources as well as individually assigned artifact packets for scoring via individual Office 365 shared folders. Each evaluator will score 12 artifacts by March 22, 2019.

 

2/20/2019

Added Michele McCarthy as scorer; email with access to materials sent.  Evaluator count at 12.  Artifact count still at 274.

 

2/13/2019

Organizational session for norming - 10 am

Members present: J. Alonso, M. McCamish, A. Kaufman, A. Wood, N. Mansito, T. Williams

Discussed timeline for norming process and scoring artifacts.  Evaluator count is at 11 currently.Artifact count is 274 currently.  Sampling plan discussed. 

Donna Seagle (Director, CARE Center) discussed a grant opportunity with AAC&U to be a part of the VALUE Institute and attend a summer conference for the team's part (100 student artifacts submitted and scored by colleagues in AAC&U and feedback given) and the potential for this process feeding into continuous improvement planning. Melanie Ribaric (HIPs Specialist) discussed the potential for developing a GCA HIPs code for courses. She is working with TBR for guidance on the process and minimum criteria.

 

Email went to scoring members at 4:00 pm with access to norming artifacts and resources.  A poll for the norming session will be sent and will be scheduled so the most scorers can attend (either Feb 27 or Mar 1).  Method is single shared Office 365 folder.

 

8/ /2018 - 1/30/2019

Recruitment of contributing instructors

 

8/ /2018 

Plenary session - 

Team members discussed 1) results from 2016-17 scoring and the work of the review year 2017-18; 2) the recruitment of instructors to contribute artifacts and 3) the use of the revised and expanded rubric for instructional and assignment design.  T. Williams shared a form that thsoe contributing artifacts need to complete which allows for 1) indication of to which area(s) of the rubric the submitted assignment aligns and 2) indication of how the instructor thinks the students SHOULD perform on the area(s) - to be used for follow-up once results are in and distributed back to instructor.  Team members will go back to respective divisions and departments to recruit volunteers.

 

2017-2018 Review Year Activities (reverse chronological order)

March 22, 2018

Summary in email from J. Alonso dated 3/23/2018

    • We all agreed it is much easier to grade assignments with the new rubric. Everyone appreciates particularly the examples for each score.
    • We also agreed the document needs a new format (the current one is very cumbersome). We decided to lighten the rubric, and create separate documents with the details for each indicator. Traci offered to create a PDF that would allow you to review the rubric and access the detail of any indicator by clicking on it. Find attached a sample of how the rubric might look like, and a sample of the detail information for three of the six indicators.  Rubric is published now the Rubrics link.
    • The score norming activity went according to plan, and we found only one major discrepancy from “Rater 7”. This rater found many “not applicable” where the other raters actually gave scores. Exploring this discrepancy further is recommended.
    • Moving forward, now it is time to recruit faculty willing to design a global and cultural awareness assignments for their fall courses. We will collect student artifacts from this assignments at the end of the fall, and work on scoring in the Spring of 2019. Here are a few basic points we discussed regarding recruitment:
      • Each one of us in the committee who teach in the HFA and SBS divisions should attempt to design and contribute a global and cultural awareness assignment
      • Each one of us in these divisions should try to recruit at least one other person to do the same
      • April is the best month to recruit, as it is ePortfolio goal-setting month. We can remind people developing this assignment is a great eportfolio goal.
      • Anyone contributing an assignment in 2018-2019 must be ready to contribute to the norming discussion in 2019-2020 (two year ePortfolio goal!)
      • When talking to prospective recruits, make sure you share the new rubric materials with them.
      • Offer prospective recruits our help to design and review the assignment: it is important the request sounds like an offer of help and collaboration, and not an imposition. 

 

Feb 22, 2018

Team members met to finalize the process of "testing" the revised rubric.  Finalized version of rubric shared with team members. Evaluation of the rubric using student artifacts was spread among team members.  Artifacts and rubric were shared via a shared folder in OneDrive.

Global Self-awarenessMichael McAmishMichael HolsombackTraci Williams
Cultural diversity, understanding differences
Knowledge, Cultural Self AwarenessTheresa LemonJuan Antonio AlonsoAngie Wood
Knowledge, Knowledge of cultural worldview frameworks
Skills, EmpathyKatheryn ThompsonRebecca AslingerNicolas Manisto
Skills,Verbal and nonverbal communication

 

Dec 8, 2017 - Jan 5, 2018

Team leads finalized the rubric.  Discussion of the process to "test" the rubric using student artifacts.  T. Williams will collect student artifacts for sharing with team members.

 

Dec 8, 2017 

Team leads (J. Alonso and M. McCamish) met with T. Williams to discuss next steps and finalization of the rubric.  The following assignments were distributed:

- Speak to Maria to come up with examples for indicator 6 (JA)

- Explain the type of assignment that would fit each indicator (JA)

- Bring up the definitions of key words to where they are relevant (JA)

- Work out the examples for indicator 1 (JA)

- Work out the empathy benchmark clarifications (JA)

- Cultural Self Awareness indicator (MM)

- Examples for Empathy (MM)

- Collaborate with the type of assignment that would fit each indicator (MM)

 

Nov 29, 2017

Team members were asked to review the lastest additions to the rubric by a visiitng team member (M. Spear) for communcation (verbal & nonverbal) as well as to offer suggestions to other areas of the rubric. (email from J. Alonso dated 11/29/2017)

 

Nov 1, 2017

Summary from email from J. Alonso dated 11/6/2017-

 

This is what we decided in our meeting last Wednesday:

 

  • We have now a description of each benchmark in the onedrive rubric document
  • One exception is the “Oral and Written Communication” indicator. We are going to reach out to Speech faculty to have them help us define the benchmarks for that indicator
  • We will have another meeting at the end of the month. In the meantime, we are to independently work in the following tasks:
    • Come up with INDICATOR definitions (last page of the onedrive document). We believe they are as important as the benchmark descriptions
    • Read everything in the document and leave your COMMENTS. Make them as detailed as possible, and do not forget to initial them
    • At the very end of the document, we are starting to collect a list of keywords that members of the team feel need further clarification or definition. If you find a term you will like to have clarified, write it at the end of the document. If you feel like you can clarify any items in that list, feel free to write a definition
    • The purpose of this meeting is to review all this work and decide on a first draft version of our new rubric
    • Once we have this draft version, we will receive some student artifacts and we will regrade them independently (through December and early January)
    • We will meet again once the Winter break is over to compare our results and bring up any issues we have encountered with the rubric draft while grading

 

 

Oct 10 - 30, 2017

Team members edited a rubric document (shared folder in OneDrive).  

 

 

Sept 28, 2017 

Team members discussed an extensive review of the rubric descriptors as well as the student performance descriptors for each criterion of the rubric. In order to maximize time, team members chose a criterion of the rubric to review before the next meeting.  Using student performance scores, sel;ected student samples, and individual evaluator experiences during scoring in 2016-17, members were asked to offer suggestions for clarifying language within the student performance levels. Materials were shared with team members via shared folderr in OneDrive.

From a reminder email from J. Alonso  dated 10/2/2017 - 

Individuals or pairs are going to take each indicator in the rubric and try to clarify in plain language what each score in that indicator implies and how it would look in a student assignment. The idea is to get back together and contrast and combine these descriptions to create a simple guide to both teach and grade global and cultural awareness. The work groups are going to be:

 

 

 

  • Global Self-Awareness: Liz Norell, Rebecca Aslinger
  • Cultural Diversity/Understanding differences: Michael Holsomback
  • Knowledge I/ Cultural self-awareness: Katheryn Thompson
  • Knowledge II/Knowledge of cultural worldview frameworks: Juan Antonio Alonso, Nick Mansito
  • Skills/Empathy: Michael McCamish, Angie Wood
  • Skills/Verbal and nonverbal communication: Traci Williams

 

 

 

DRAFT: This module has unpublished changes.