[sakai-dev] Re: [sakai-user] Who is running GradebookNG in production? Status?

classic Classic list List threaded Threaded
1 message Options
Shawn Foster-2 Shawn Foster-2
Reply | Threaded
Open this post in threaded view
|

[sakai-dev] Re: [sakai-user] Who is running GradebookNG in production? Status?

Hi, Laura.

I wanted to chime in with what we did when we upgraded to Sakai 11.

TL;DR:

We are running a locally-enhanced version of GradebookNG (GBNG), which
we call GradebookNG+ (GBNG+). We created this local version after we
identified issues and limitations with the original GBNG during our
upgrade to Sakai 11 last year (that I will outline below). We have been
running our GBNG+ for over four months in production and have over 4000
published course sites using it this year. It has been working well, now
that we've addressed most of our original issues with GBNG.

I will try to answer most of Laura's questions along the way.

-----

We at Western University upgraded to Sakai 11.3 this past August and
have been running GBNG+ in production as the only Gradebook in our local
Sakai instance. We made the decision during our upgrade process to Sakai
11 to only run one Gradebook to reduce support confusion and because of
our prior local customizations of Gradebook: we didn't want to have to
port those customizations twice - once to each Gradebook. (Our local
customizations to the Gradebook include displaying student numbers in
the Gradebook, a custom import process for Scantron exams, and a custom
final grade approval and export process to our SIS.)

So, contrary to most people's recommendations, we stealthed Gradebook
Classic and have been running GradebookNG+ by itself for all sites (new
and old) for over a term now. I just ran a report last week that says we
have over 4000 published course sites for this year's Fall and Winter
terms using GBNG+. I am happy with our decision to only run one
Gradebook, but it took a lot of work to get us to this point, which I
will describe below.

We upgraded to Sakai 11.3 in August 2017 after an eight-month upgrade
process starting around this time last year. That process included
updating our development environments, porting our local customizations,
doing a comparative evaluation between our local Sakai 10 instance and
stock Sakai 11.3 (to better understand what had changed), testing
multiple rounds of QA, and re-writing a significant portion of GradebookNG.

Beginning in March, we noticed that there were performance issues and
limitations to GBNG, such as:
- the student filter only found students on the current page
- the spreadsheet lagged for large classes with lots of students and
lots of columns
- the spreadsheet lagged for most Gradebooks when viewed with non-Chrome
browsers
- there was no option available through the interface to adjust the page
size of the spreadsheet (number of student rows displayed)
- the import process took hours to complete (one import test from a
production course site's Gradebook took 12-13 hours to finish; an
empty-column import from a fresh export took 11 minutes)
- AJAX requests were refreshing the whole page instead of just the
particular change - causing unnecessary page reloads
- the horizontal scrollbar was at the bottom of the speadsheet and not
easily discoverable for users with non-horizontal scrolling capability
- the category headings changed to different random colours every time
the Gradebook is re-loaded, which could have been confusing to users
associating a particular colour to a particular category
- the spreadsheet-like interface was not useable on narrow screens, such
as mobile devices, because of the fixed columns
- the import process had issues with validation, didn't accept
mathematical characters (such as hyphens in titles), and caused data loss
- the rows in the fixed columns on the left could get mis-aligned from
their adjacent item rows

Some of these concerns were specific for course sizes around or above
100 students. We, therefore, ran some queries to determine the number
and sizes of our courses. We found that our largest course site had 2114
students. We had over 2400 sites with more than 100 students. Over 500
sites containing more than 100 students had been created in year prior.

(To answer your #2 question, Laura, we have many course sites over 1000
students. I am aware of gradebooks with 25 to 50 items. The one that I
used for the import test that I mentioned above had 72 items with over
1000 students. This is common for our first-year introductory courses
across disciplines. Also as I mentioned earlier, we had a custom final
grade approval and export process prior to Sakai 11. We were able to
redesign it to work with GBNG+. It auto-rounds course grades to whole
numbers, allows Instructors and Grade Admins to override and review, and
provides an approval process to securely transmit the final grades
digitally to our Registrar office.)

Around the time we identified these issues last March, we became aware
of the "Gradebook table refactor", which was supposed to address some of
the issues that we had discovered. However, the details were sparse and
the timeline was uncertain (or, at best, tight for our upgrade's
schedule). We, therefore, made the decision to address the issues
ourselves.

We rewrote the spreadsheet markup so it used one HTML table (instead of
three overlapping tables used in GBNG), which helped improve the
performance. We also:
- added a student filter for all students
- implemented standard Sakai table paging with page sizing
- significantly improved the import speed (that 12-to-13-hour import
took about 97 seconds after the improvements)
- AJAX'd most requests that made sense on the main page to reduce the
number of page reloads
- redesigned the size of the spreadsheet, so it fits on the screen all
at once to make the scrollbars more obvious
- made the category headers colour deterministic, so they wouldn't
change colours (and the colour wouldn't have to be stored in the database)
- removed the fixed columns in narrow-width views, so the spreadsheet
was still usable on mobile devices
- improved the import process to better handle special cases and
decision changes
- reduced the column width in the spreadsheet interface to allow more
columns to be shown on the screen at one time

I've created a video of our Gradebook to give a brief overview of what
we released in August as GBNG+:

https://owl.uwo.ca/x/lDKUUV
(best viewed with Firefox or Chrome)

Since GBNG has already moved forward with the "Gradebook table refactor"
and the use of Handsontable for that, we'll have to evaluate whether our
issues have been addressed for Sakai 12. We are cautiously optimistic
about the performance improvements of a table that uses infinite
scrolling: that can be a lot of data on the page at once for big courses
and browsers could still chug and jitter. We'll see. Also, getting to
the final rows or pages of the Gradebook in GBNG+ is easy (just hit the
>| paging button, like in other Sakai tools) compared to having to
scroll through 1000 or 2000 student rows.

My message may appear to be overly negative, but I don't want it to be:
the work that the GradebookNG team did with the user testing, redesign,
and end product for Sakai 11's GBNG is a huge leap forward for the
Gradebook tool and for Sakai. Most tools could use this type of redesign
project to modernize them. I want to congratulate and thank all of the
people who were involved on the GBNG project for Sakai 11, including
(but not limited to) Steve, Payten, Mark, Kyle, and Jeff. Thanks, everyone!

My response above to Laura's message was to share our experiences about
what we discovered and encountered, which may be useful to others as
they make the switch to GradebookNG. I feel sorry for anyone using
GradebookNG in Sakai 11 because GradebookNG is not designed for classes
bigger than 100 students. Hopefully it will better for large classes in
Sakai 12.

The community will need to expand the QA test cases and QA data to more
accurately reflect large production use cases for Sakai 12 and for
future projects - particularly for large-scale institutions, like Western.

We (at Western) are evaluating options locally to determine how we can
be involved in the Sakai upgrade process earlier while still meeting our
local commitments and obligations.

Thank you if you made it to the bottom of this message. I hope this
information was useful to you.

-Shawn

--
Shawn Foster
eLearning Technology and Application Development
Western Technology Services
Western University
London, Ontario, Canada


On 2017-12-13 12:05 PM, Laura Gekeler wrote:

> The University of Notre Dame is running GradebookNG in production on our
> 11.4 Sakai instance.
> We are also running Gradebook2.
>
> It's been our hope that that 2017-2018 academic year would be an overlap
> year for the two Gradebooks, but that at the end of Spring 2018 we can
> decommission Gradebook2 in favor of Gradebook NG.
>
> 1. Who else is running GradebookNG in production?
> 2. What kinds Gradebooks are we running? Do they hit boundaries or confirm
> no boundaries exist?
>
>    - # of students in Gradebook? (We have one running 254 students. )
>    - # of grade items in the Gradebook
>    - Resiliency: Do we know of instructors who change settings mid term? Do
>    the calculations tolerate being flipped from points to percentages or
>    adding weighting or dropping scores?
>    - Exporting Grades for import to SIS's - Is anyone doing this? How are
>    the exports working?
>    - Exporting Grade to ReImport New Column: Is anyone doing this? How are
>    the exports working?
>
>
> To answer my own questions:
>
> 2. Kinds of Gradebooks, here's one example: Settings are: Points,
> Categories, No Weighting, but with an Extra Credit category. Uses the
> Letter Grade schema.
>
>    - 244 students. 100 fit on the screen at any given time. <<<>>> chevrons
>    seem to function only intermittently (Sakai 12 GBNG loads all students on
>    single page, seems like a good idea).
>    - 42 items in the GB, doesn't seem excessive to me. Others may have more?
>    - Exporting grades for import to SIS : we've had a couple do it with no
>    problems
>    - Exporting grades for re-import with new grade column back to Sakai:
>    seems a little touchy but I don't have exact steps.
>
>
> Thanks for any feedback others may share!
>
> Laura
>
>
> Laura Gekeler
> LMS Administrator
> Learning Platforms
> Apereo Foundation <https://www.apereo.org/content/projects-communities>
> Board '14-2020
> University of Notre Dame
> [hidden email]
>

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.