Session VIII Characteristics of Good SelfStudy Reports Tom Tanner ATS Commission Staff Before we begin a caution ATS Commission on Accrediting 2015 SelfStudy Workshop If youre going to work that hard do the right car ID: 752429
Download Presentation The PPT/PDF document "2015 Self-Study Workshop" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
2015 Self-Study Workshop
Session VIII
Characteristics of
Good Self-Study Reports
Tom Tanner
ATS Commission StaffSlide2
Before we begin, a caution…
ATS Commission on Accrediting: 2015 Self-Study Workshop
If you’re going to work that hard, do the right car…Slide3
Characteristics of Good Self-Study Reports
Five things to remember to do the “right” report:
The 4 purposes of the report
The 3 qualities of the report
The 5 features of good/weak reports
The 4 parts of the report
The 3 last things to remember
ATS Commission on Accrediting: 2015 Self-Study Workshop
1
2
3
4
5Slide4
Characteristics of Good Self-Study Reports
Five things to remember to do the “right” report:
The 4 purposes of the report
The 3 qualities of the report
The 5 features of good/weak reportsThe 4 parts of the report
The 3 last things to remember
ATS Commission on Accrediting: 2015 Self-Study Workshop
1Slide5
Characteristics of Good Self-Study Reports
The 4 purposes of the report
(www.ats.edu)
ATS Commission on Accrediting: 2015 Self-Study Workshop
Tab 5
(page 7)Slide6
Characteristics of Good Self-Study Reports
The 4 purposes of the report
Demonstrate
ability to analyze institution’s own effectiveness
Develop plans for improvement
Provide
evidence of meeting standards
Form basis for external review
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide7
Characteristics of Good Self-Study Reports
The 4 purposes of the report
Demonstrate
ability to analyze institution’s own effectiveness
Good long look in the mirrorThe mirror has two faces: School’s own
mission
ATS Commission standards
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide8
Characteristics of Good Self-Study Reports
The 4 purposes of the report
Develop
plans for improvement
Report must not only “look around” but “look ahead” Assessment (self-study) should show
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide9
Characteristics of Good Self-Study Reports
The 4 purposes of the report
Provide
evidence of meeting standards
Weak report: “Our seminary clearly meets this standard.”
Good
report: “
Evidence of how our seminary meets this standard may be summarized in these three ways. First,…”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide10
Characteristics of Good Self-Study Reports
The 4 purposes of the report
Form
basis for external review
The self-study report is your story…
but
told to strangers
(so don’t assume) who are sympathetic, but objective so you must
present and persuade
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide11
Characteristics of Good Self-Study Reports
Five things to remember to do the “right” report:
The 4 purposes of the report
The 3 qualities of the report
The 5 features of good/weak reportsThe 4 parts of the report
The 3 last things to remember
ATS Commission on Accrediting: 2015 Self-Study Workshop
2Slide12
Characteristics of Good Self-Study Reports
The 3 qualities of the report*
Provide sufficient
description
Document appropriate evaluationIdentify strategic recommendations
*
Page 11 of
Chapter Two of the Handbook
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide13
Characteristics of Good Self-Study Reports
The 3 qualities of the report
Provide sufficient
description
Weak report: “We have sufficient faculty, and they are all sufficiently qualified.”
Good
report: “We have seven full-time faculty; six have PhD degrees and one has a DMin (see
Faculty Credentials for detailed list).”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide14
Characteristics of Good Self-Study Reports
The 3 qualities of the report
Provide
sufficient
descriptionWeak
report:
“
Our SLAP efforts have produced significant improvements.”Good report:
“Our Student Learning Assessment Program (affectionately called SLAP by students) has produced these three improvements in student learning this past year. First,...”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide15
Characteristics of Good Self-Study Reports
The 3 qualities of the report
Provide sufficient
descriptionDocument appropriate evaluation
Identify strategic recommendations
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide16
Characteristics of Good Self-Study Reports
The 3 qualities of the report
Document appropriate
evaluation
Weak report: “Our MDiv program addresses the ‘cultural context’ requirement (Degree Program Standard A, section A.2.3) fairly well, but could use improvement.”
Good
report: “We evaluate the ‘cultural context’ component
in two ways: (1) a cultural awareness survey (indirect measure) and (2) a cultural engagement project (direct
measure). Results are provided in Table A.2 below, followed by a discussion of how those results have been used to improve our MDiv students’ learning in this critical area.”ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide17
Characteristics of Good Self-Study Reports
The 3 qualities of the report
Document appropriate
evaluation
Weak report: “We try hard not to spend more than we take in.”
Good
report: “The seminary has balanced its budget four of the last five years, as Table 8.6 below demonstrates. The deficit of $74,892 in FY12 represented only a 2.5% overage due to an unexpected need to replace a furnace. We have since established a ‘contingency fund’ for such emergencies, with a line-item budget of $50,000 per year.”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide18
Characteristics of Good Self-Study Reports
The 3 qualities of the report
Provide sufficient
descriptionDocument appropriate evaluation
Identify strategic recommendations
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide19
Characteristics of Good Self-Study Reports
The 3 qualities of the report
Identify strategic recommendations
Weak
report: “
We know more should be done to address our lack of board diversity, and we are working on that.”
Good
report: “Given our global mission and given that one fourth of our students and staff are ethnic minorities, while no one on the board is, our trustees have set a goal of recruiting one new ethnic trustee each year for the next five years until our board more fairly represents our constituency and our mission. Our first African American trustee will begin her term this fall.”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide20
Characteristics of Good Self-Study Reports
The 3 qualities of the report
Identify strategic recommendations
Weak
report: “As you will see scattered throughout this report, we have identified dozens and dozens of different areas to improve.”
Good
report: “We have organized each chapter of the report (one per standard) into four parts: (1) a brief description of that area, (2) a critical evaluation of that area, (3) a list of strengths in that area, and (4) a set of two or three strategic recommendations per area to address key concerns identified. All recommendations are summarized and prioritized in the last chapter.”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide21
Characteristics of Good Self-Study Reports
Five things to remember to do the “right” report:
The 4 purposes of the report
The 3 qualities of the report
The 5 features of good/weak reportsThe 4 parts of the report
The 3 last things to remember
ATS Commission on Accrediting: 2015 Self-Study Workshop
3Slide22
Characteristics of Good Self-Study Reports
The 5 features of
good
[vs. weak] reports
Are organized coherently
Describe the self-study process
Present the data crucial to understanding
Comprehensive as necessary; brief as possible
Demonstrate objectivity and honesty
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide23
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Are organized coherentlyMust be organized by the standards
If joint/coordinated visit, must have detailed “cross walk” [Joint visits only with NEASC; coordinated, possible with any]
Must also have Introduction and Conclusion Will discuss later under “The 4 parts of the report”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide24
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Describe the self-study processIntroduction describes process
Process
engages the community
Reflected in report and interviews
Caution: Summarize process in 1-2 pagesIf use people’s names, use titles too
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide25
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Present the data crucial to understandingBe clear and concise with data
Use qualitative and quantitative data
Use tables and charts to support narrative
Caution: Put extensive data in Documents Room/links
But include summary data in text When using links, make easy to use
ATS Commission on Accrediting: 2015 Self-Study Workshop
Clearly label each itemSlide26
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Present the data crucial to understanding
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide27
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Present the data crucial to understanding
Weak
report:
“Our students are very diverse.”Good report:
“Our 150 students, whose ages range from 24 to 64 (with most in their 30s), are 45% female, and 30% ethnic minorities, with a dozen different denominations represented (see Table 6.3 below for demographic details). This is what one recent graduate said about her experience here regarding diversity: ‘[quote].’”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide28
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Present the data crucial to understandingWeak
report:
“Most of our MDiv students are placed.”Good report:
“Over the last five years our MDiv graduates have averaged a 90% placement rate, with a steady increase each year, as the table below shows.”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide29
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Present the data crucial to understandingWeak
report:
“We evaluated our student services through a survey last year; see data at this link.”
Good report: “Faculty and staff reviewed the results of a recent student survey (2014 Student Survey,
linked here and also in File 6.2 in Documents Room:) and drew these four conclusions regarding student services…”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide30
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Comprehensive as necessary; brief as possibleWeak
reports:
Missing standards
(e.g., Degree Program Standards)Too few pages: 50 pages won’t cover the standards
Too many pages: 175+ pages overwhelm reviewersPrinted incorrectly: Single-sided/double-spaced reports double the length and the weight (remember your traveling readers)
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide31
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Comprehensive as necessary; brief as possibleGood
reports:
100-135 pages is “ideal range”
5-10 page Introduction75ish pages for General Institutional Standards (1-8)
15-40 pages for Educational and Degree Program Standards5-10 page ConclusionDon’t overwhelm text with supplemental information
Put lengthy information in appendices or links
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide32
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Demonstrate objectivity and honestyWeak report:
“Morale on campus is much better.”
Good report:
“Three years ago faculty morale hit a low point when a doctrinal controversy led to the dismissal of a popular professor. The seminary followed its procedures (documented in our personnel files), but it was a very difficult time. Since then, the board and administration have worked hard to rebuild unity, with significant success—as noted in the recent faculty survey (see 2015 Faculty Survey
, item #8).”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide33
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Demonstrate objectivity and honestyWeak report:
“The seminary has experienced some administrative turnover since the last visit.”Good report
: “We have had three presidents and four deans in the last decade. These turnovers have occurred for various reasons and with varying results. We will summarize those reasons and results here, but the evaluation committee is also encouraged to ask about this during their on-site interviews to have a more nuanced understanding of this issue.”
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide34
Characteristics of Good Self-Study Reports
The 5 features of
good/weak
reports
Demonstrate objectivity and honestyWeak report:
[no mention of recent public controversy]
Good report: “
Just prior to submitting this self-study report to our trustees for their review, five students filed a complaint against a faculty member alleging sexual harassment. Unfortunately, portions of that complaint have surfaced recently in social media. We have initiated our formal complaint process, and we will inform the evaluation committee of any significant developments that we are permitted legally and ethically to share.”
NOTE: Boards are to “receive” self-study reports (p. 10 of Chapter Two in Handbook)
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide35
Characteristics of Good Self-Study Reports
Five things to remember to do the “right” report:
The 4 purposes of the report
The 3 qualities of the report
The 5 features of good/weak reportsThe 4 parts of the report
The 3 last things to remember
ATS Commission on Accrediting: 2015 Self-Study Workshop
4Slide36
Characteristics of Good Self-Study Reports
The 4 parts of the report*
Introduction
Main Narrative
ConclusionAppendices (do not print; on flash drive only)
*
Pages 13-14
of Chapter Two of the Handbook
[NOTE: These 4 parts were discussed, in part, earlier]
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide37
Characteristics of Good Self-Study Reports
The 4 parts of the report
Introduction
Main Narrative
ConclusionAppendices
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide38
Characteristics of Good Self-Study Reports
The 4 parts of the report
Introduction
(5-10 pages)
Introduce the school (mission/history)
Review accreditation history
(since last visit)
Give overview of the self-study processIdentify any special features of the report
Summarize any major changes (since last visit)Orient readers to any unique school issues
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide39
Characteristics of Good Self-Study Reports
The 4 parts of the report
Main Narrative
(90-115 pages)Part 1: General Institutional Standards (75 pp.)
Standards 1-8—must cover each one
Part 2: Educational and Degree Program Standards (15-40 pp.)
Educational Standard—cover only applicable sections
Degree Programs Standards—cover only applicable standardsNOTE: Can cover assessment under ES.6 or under section 1 of each DPS
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide40
Characteristics of Good Self-Study Reports
The 4 parts of the report
Main Narrative
(90-115 pages)One suggested structure per chapter (per earlier slide):
brief description
of school’s resources for that standard
critical evaluation
of school’s alignment with standard
list of strengths of the school in terms of that standardstrategic recommendations to improve in that standardNOTE: If not meeting standard, must describe plans to do so.
* Do not need to cover every subsection in every standard: group things.
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide41
Characteristics of Good Self-Study Reports
ATS Commission on Accrediting: 2015 Self-Study Workshop
Weak
Report:
Discuss each of
10 subsections
Good
Report:
Discuss each of 3 sections
Standard 4 Library (as example)Slide42
Characteristics of Good Self-Study Reports
The 4 parts of the report
Conclusion
(5-10 pages)Summarize overall findings of self-studyOrganize and prioritize
strengths
Organize and prioritize
concernsOrganize and prioritize recommendations
Conclude with desired action (optional)
ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide43
Characteristics of Good Self-Study Reports
The 4 parts of the report
Appendices
(For any other material, provide link in report)
Required appendix materials (bookmarked PDF on USB)
1)
Current org chart 5) Most recent audit and letter
2) Current strategic plan 6) Handbooks (board, staff, etc.)3) Current assessment plan 7) Academic catalog
4) Current budget 8) Targeted Issues Checklist* See list on pages 16-17 of Chapter Three of the
Handbook** Also need Word version of TIC for committee to complete
ATS Commission on Accrediting: 2015 Self-Study Workshop
**
*Slide44
Characteristics of Good Self-Study Reports
The 4 parts of the report
Items for Documents Room (not in appendix)
See list on pages 17-18 of Chapter Three of Handbook
Documents Room may be physical or virtual
(if virtual, see guidelines on pages 15-16 of Chapter Three)
These and other items may be linked in report
“other items” such as Bylaws, Charter, etc. ATS Commission on Accrediting: 2015 Self-Study WorkshopSlide45
Characteristics of Good Self-Study Reports
Five things to remember to do the “right” report:
The 4 purposes of the report
The 3 qualities of the report
The 5 features of good/weak reportsThe 4 parts of the report
The 3 last things to remember
ATS Commission on Accrediting: 2015 Self-Study Workshop
5Slide46
Characteristics of Good Self-Study Reports
5. The 3 last things to remember
Tell
your story in light of the standards
to improve your students and school
ATS Commission on Accrediting: 2015 Self-Study Workshop
Where will this journey take you?