[sakai-dev] Samigo autoscore null for some answers

classic Classic list List threaded Threaded
7 messages Options
Austin Nakasone Austin Nakasone
Reply | Threaded
Open this post in threaded view
|

[sakai-dev] Samigo autoscore null for some answers

Hello Sakai Devs,

Has anyone ever seen anything in Samigo (tests & quizzes) where after a student submits their answers some of their answers' autoscore remain NULL?  Even though they answered correctly?

When this happens, the instructor's UI shows that they got the question correct but the score is 0 (which I assume is because of the NULL autoscore value)

I didn't see anything in JIRA that sounded similar (though maybe I wasn't searching the right keywords)

for example, here's what it looks like in the database.  All of the questions except the last one have an autoscore.  what's also strange is that the 2nd to the last one has a NULL overridescore.

mysql> select SUBMITTEDDATE, PUBLISHEDANSWERID, RATIONALE, ANSWERTEXT, AUTOSCORE, OVERRIDESCORE, comments from sam_itemgrading_t where assessmentgradingid = '2364664' order by submitteddate;
+---------------------+-------------------+-----------+------------+-----------+---------------+----------+
| SUBMITTEDDATE       | PUBLISHEDANSWERID | RATIONALE | ANSWERTEXT | AUTOSCORE | OVERRIDESCORE | comments |
+---------------------+-------------------+-----------+------------+-----------+---------------+----------+
| 2018-01-12 08:55:16 |          12318602 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 08:58:01 |          12318625 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 08:58:05 |          12318631 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:00:22 |          12318576 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:00:25 |          12318618 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:00:35 |          12318585 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:01:00 |          12318590 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:01:25 |          12318634 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:01:45 |          12318627 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:02:17 |          12318641 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:02:41 |          12318614 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:03:27 |          12318637 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:05:33 |          12318594 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:06:28 |          12318612 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:07:04 |          12318620 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:08:48 |          12318600 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:09:14 |          12318572 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:09:50 |          12318580 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:12:52 |          12318608 | NULL      | NULL       |         1 |          NULL | NULL     |
| 2018-01-12 09:15:06 |          12318644 |           | NULL       |      NULL |          NULL | NULL     |
+---------------------+-------------------+-----------+------------+-----------+---------------+----------+
20 rows in set (0.01 sec)

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.
Austin Nakasone Austin Nakasone
Reply | Threaded
Open this post in threaded view
|

[sakai-dev] Re: Samigo autoscore null for some answers

Hello All,

Just wondering if anyone has any idea about this problem?  (btw, we're using Sakai 11.4)

I was thinking that this could be due to https://jira.sakaiproject.org/browse/SAM-3191 and maybe it could be, but the fact that the autoscore is null for that one question (as well as null overridescore for two questions), kind of bothers me... what would cause them to be null in the first place?

Thanks,

Austin

On Wed, Jan 24, 2018 at 12:11 PM, Austin <[hidden email]> wrote:
Hello Sakai Devs,

Has anyone ever seen anything in Samigo (tests & quizzes) where after a student submits their answers some of their answers' autoscore remain NULL?  Even though they answered correctly?

When this happens, the instructor's UI shows that they got the question correct but the score is 0 (which I assume is because of the NULL autoscore value)

I didn't see anything in JIRA that sounded similar (though maybe I wasn't searching the right keywords)

for example, here's what it looks like in the database.  All of the questions except the last one have an autoscore.  what's also strange is that the 2nd to the last one has a NULL overridescore.

mysql> select SUBMITTEDDATE, PUBLISHEDANSWERID, RATIONALE, ANSWERTEXT, AUTOSCORE, OVERRIDESCORE, comments from sam_itemgrading_t where assessmentgradingid = '2364664' order by submitteddate;
+---------------------+-------------------+-----------+------------+-----------+---------------+----------+
| SUBMITTEDDATE       | PUBLISHEDANSWERID | RATIONALE | ANSWERTEXT | AUTOSCORE | OVERRIDESCORE | comments |
+---------------------+-------------------+-----------+------------+-----------+---------------+----------+
| 2018-01-12 08:55:16 |          12318602 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 08:58:01 |          12318625 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 08:58:05 |          12318631 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:00:22 |          12318576 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:00:25 |          12318618 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:00:35 |          12318585 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:01:00 |          12318590 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:01:25 |          12318634 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:01:45 |          12318627 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:02:17 |          12318641 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:02:41 |          12318614 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:03:27 |          12318637 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:05:33 |          12318594 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:06:28 |          12318612 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:07:04 |          12318620 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:08:48 |          12318600 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:09:14 |          12318572 |           | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:09:50 |          12318580 | NULL      | NULL       |         1 |             0 | NULL     |
| 2018-01-12 09:12:52 |          12318608 | NULL      | NULL       |         1 |          NULL | NULL     |
| 2018-01-12 09:15:06 |          12318644 |           | NULL       |      NULL |          NULL | NULL     |
+---------------------+-------------------+-----------+------------+-----------+---------------+----------+
20 rows in set (0.01 sec)


--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.
John Ansorge John Ansorge
Reply | Threaded
Open this post in threaded view
|

RE: [sakai-dev] Re: Samigo autoscore null for some answers

I made the SAM-3191 patch, although it’s been a while so my memory is rather hazy.  I’m pretty sure that what you’re describing is a different bug since it seems to affect “Multiple Choice, Single Selection” questions.  If I recall correctly the change in SAM-3191 is within a block of code only run for “Multiple Choice, Multiple Correct” questions so it would not do anything to fix the other kinds of multiple choice questions. 

 

But it is certainly possible there’s another point in the code where two floats are being compared with too much precision for MC,SS questions, leading to a similar symptom. 

 

John

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Sanghyun Grace Jeon
Sent: Wednesday, February 07, 2018 1:05 PM
To: 'Austin' <[hidden email]>; sakai-dev <[hidden email]>
Subject: RE: [sakai-dev] Re: Samigo autoscore null for some answers

 

Dear Austin,

 

We are using Sakai 11.2 and SAM-3191 issue bit us this week. Student received 0, although s/he selected all correct answers. I just checked our database again, but I don’t see any null value. All 0 is recorded for totalautoscore, totaloverridescore and finalscore. We are using Mysql 5.5.56

 

S

 

From: [hidden email] [[hidden email]] On Behalf Of Austin
Sent: Wednesday, February 7, 2018 12:49 PM
To: sakai-dev <[hidden email]>
Subject: [sakai-dev] Re: Samigo autoscore null for some answers

 

Hello All,

 

Just wondering if anyone has any idea about this problem?  (btw, we're using Sakai 11.4)

 

I was thinking that this could be due to https://jira.sakaiproject.org/browse/SAM-3191 and maybe it could be, but the fact that the autoscore is null for that one question (as well as null overridescore for two questions), kind of bothers me... what would cause them to be null in the first place?

 

Thanks,

 

Austin

 

On Wed, Jan 24, 2018 at 12:11 PM, Austin <[hidden email]> wrote:

Hello Sakai Devs,

 

Has anyone ever seen anything in Samigo (tests & quizzes) where after a student submits their answers some of their answers' autoscore remain NULL?  Even though they answered correctly?

 

When this happens, the instructor's UI shows that they got the question correct but the score is 0 (which I assume is because of the NULL autoscore value)

 

I didn't see anything in JIRA that sounded similar (though maybe I wasn't searching the right keywords)

 

for example, here's what it looks like in the database.  All of the questions except the last one have an autoscore.  what's also strange is that the 2nd to the last one has a NULL overridescore.

 

mysql> select SUBMITTEDDATE, PUBLISHEDANSWERID, RATIONALE, ANSWERTEXT, AUTOSCORE, OVERRIDESCORE, comments from sam_itemgrading_t where assessmentgradingid = '2364664' order by submitteddate;

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

| SUBMITTEDDATE       | PUBLISHEDANSWERID | RATIONALE | ANSWERTEXT | AUTOSCORE | OVERRIDESCORE | comments |

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

| 2018-01-12 08:55:16 |          12318602 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 08:58:01 |          12318625 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 08:58:05 |          12318631 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:22 |          12318576 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:25 |          12318618 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:35 |          12318585 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:00 |          12318590 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:25 |          12318634 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:45 |          12318627 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:02:17 |          12318641 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:02:41 |          12318614 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:03:27 |          12318637 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:05:33 |          12318594 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:06:28 |          12318612 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:07:04 |          12318620 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:08:48 |          12318600 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:09:14 |          12318572 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:09:50 |          12318580 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:12:52 |          12318608 | NULL      | NULL       |         1 |          NULL | NULL     |

| 2018-01-12 09:15:06 |          12318644 |           | NULL       |      NULL |          NULL | NULL     |

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

20 rows in set (0.01 sec)

 

 

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.
Austin Nakasone Austin Nakasone
Reply | Threaded
Open this post in threaded view
|

Re: [sakai-dev] Re: Samigo autoscore null for some answers

I wonder if this is a new Samigo bug introduced in Sakai 11?

I checked our DB for assessments with null 'autoscore' prior to us upgrading to Sakai 11.4 and there were none.

mysql> select assessmentgradingid, SUBMITTEDDATE, PUBLISHEDANSWERID, RATIONALE, ANSWERTEXT, AUTOSCORE, OVERRIDESCORE, comments from sam_itemgrading_t where submitteddate > "2017-09-01 00:00:00" and submitteddate < "2017-12-01 00:00:00"and autoscore is null group by assessmentgradingid;
Empty set (3 min 7.57 sec)

also, since upgrading, there have been 35 instances of samigo assessments with null autoscores:

mysql> select count(*) from (select assessmentgradingid from sam_itemgrading_t where submitteddate > "2018-01-01 00:00:00" and autoscore is null group by assessmentgradingid) as t;
+----------+
| count(*) |
+----------+
|       34 |
+----------+
1 row in set (6 min 6.16 sec)


Then checking our tomcat logs, these errors that happened in the minutes after the bad scoring, I'm seeing these errors:

04-Feb-2018 21:37:12.495 ERROR [ajp-bio-8009-exec-11] org.sakaiproject.tool.assessment.services.GradingService.storeGrades unable to retrive itemDataIfc for: null
04-Feb-2018 21:37:12.495 ERROR [ajp-bio-8009-exec-11] org.sakaiproject.tool.assessment.services.GradingService.storeGrades unable to retrive itemDataIfc for: null


unfortunately, I still don't know how to reproduce the problem, but if those errors in the logs are related, searching the code base for those errors, I see:

GradingService.java
(line 937)
        itemId = itemGrading.getPublishedItemId();
        ItemDataIfc item = (ItemDataIfc) publishedItemHash.get(itemId);
        if (item == null) {
        //this probably shouldn't happen
        log.error("unable to retrive itemDataIfc for: {}", publishedItemHash.get(itemId));
        continue;
        }

(line 1048)
      //SAM-1724 it's possible the item is not in the hash -DH
        if (item == null) {
        log.error("unable to retrive itemDataIfc for: " + publishedItemHash.get(itemId));
        continue;
        }


If this indeed the problem, what would cause the item to not be in the hash?

Thanks,

Austin





On Wed, Feb 7, 2018 at 12:53 PM, John Ansorge <[hidden email]> wrote:

I made the SAM-3191 patch, although it’s been a while so my memory is rather hazy.  I’m pretty sure that what you’re describing is a different bug since it seems to affect “Multiple Choice, Single Selection” questions.  If I recall correctly the change in SAM-3191 is within a block of code only run for “Multiple Choice, Multiple Correct” questions so it would not do anything to fix the other kinds of multiple choice questions. 

 

But it is certainly possible there’s another point in the code where two floats are being compared with too much precision for MC,SS questions, leading to a similar symptom. 

 

John

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Sanghyun Grace Jeon
Sent: Wednesday, February 07, 2018 1:05 PM
To: 'Austin' <[hidden email]>; sakai-dev <[hidden email]>
Subject: RE: [sakai-dev] Re: Samigo autoscore null for some answers

 

Dear Austin,

 

We are using Sakai 11.2 and SAM-3191 issue bit us this week. Student received 0, although s/he selected all correct answers. I just checked our database again, but I don’t see any null value. All 0 is recorded for totalautoscore, totaloverridescore and finalscore. We are using Mysql 5.5.56

 

S

 

From: [hidden email] [[hidden email]] On Behalf Of Austin
Sent: Wednesday, February 7, 2018 12:49 PM
To: sakai-dev <[hidden email]>
Subject: [sakai-dev] Re: Samigo autoscore null for some answers

 

Hello All,

 

Just wondering if anyone has any idea about this problem?  (btw, we're using Sakai 11.4)

 

I was thinking that this could be due to https://jira.sakaiproject.org/browse/SAM-3191 and maybe it could be, but the fact that the autoscore is null for that one question (as well as null overridescore for two questions), kind of bothers me... what would cause them to be null in the first place?

 

Thanks,

 

Austin

 

On Wed, Jan 24, 2018 at 12:11 PM, Austin <[hidden email]> wrote:

Hello Sakai Devs,

 

Has anyone ever seen anything in Samigo (tests & quizzes) where after a student submits their answers some of their answers' autoscore remain NULL?  Even though they answered correctly?

 

When this happens, the instructor's UI shows that they got the question correct but the score is 0 (which I assume is because of the NULL autoscore value)

 

I didn't see anything in JIRA that sounded similar (though maybe I wasn't searching the right keywords)

 

for example, here's what it looks like in the database.  All of the questions except the last one have an autoscore.  what's also strange is that the 2nd to the last one has a NULL overridescore.

 

mysql> select SUBMITTEDDATE, PUBLISHEDANSWERID, RATIONALE, ANSWERTEXT, AUTOSCORE, OVERRIDESCORE, comments from sam_itemgrading_t where assessmentgradingid = '2364664' order by submitteddate;

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

| SUBMITTEDDATE       | PUBLISHEDANSWERID | RATIONALE | ANSWERTEXT | AUTOSCORE | OVERRIDESCORE | comments |

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

| 2018-01-12 08:55:16 |          12318602 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 08:58:01 |          12318625 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 08:58:05 |          12318631 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:22 |          12318576 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:25 |          12318618 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:35 |          12318585 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:00 |          12318590 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:25 |          12318634 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:45 |          12318627 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:02:17 |          12318641 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:02:41 |          12318614 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:03:27 |          12318637 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:05:33 |          12318594 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:06:28 |          12318612 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:07:04 |          12318620 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:08:48 |          12318600 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:09:14 |          12318572 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:09:50 |          12318580 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:12:52 |          12318608 | NULL      | NULL       |         1 |          NULL | NULL     |

| 2018-01-12 09:15:06 |          12318644 |           | NULL       |      NULL |          NULL | NULL     |

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

20 rows in set (0.01 sec)

 

 

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.


--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.
David Hutchins David Hutchins
Reply | Threaded
Open this post in threaded view
|

Re: [sakai-dev] Re: Samigo autoscore null for some answers

I posted to this list a few weeks back, in regards to similar issues.

[sakai-dev] SAM-1724 still a production issue.
>
> Greetings Devs,
>   We have encountered this issue covered up in SAM-1724 a few times recently. The work done in that issue effectively prevents any exception being thrown, and generates a fairly useless log message. The issue as we have seen it present itself is that > the grade in Samigo will be fine, but a 0 will get pushed to the gradebook. Of course there is no indication to the student or instructor that anything has gone wrong, other than the 0 grade in the gradebook, with the message "unable to retrive itemDataIfc > for: null" being logged. Due to the way the log message is implemented it always logs "null" instead of the ItemId. I assume that there is some sort of session state corruption at play, but as of yet have been unable to reproduce this in a dev environment. 
> My question the the group is if anyone has any way to reproduce, or other ideas that could help get to the core of this issue and actually fix it, rather than just covering it up?
> Thanks:
>    -David

The logging is broken for sure, but I would be curious to find a solid way to reproduce, and maybe understand a bit better how the issue is encountered. In our production logs, there does seem to be a correlation with the instructor working in the grading screen (in samigo, not GB) while the student submits. I have not been able to confirm if that is a cause, or just coincidental though. 

On Tue, Feb 13, 2018 at 2:37 PM, Austin <[hidden email]> wrote:
I wonder if this is a new Samigo bug introduced in Sakai 11?

I checked our DB for assessments with null 'autoscore' prior to us upgrading to Sakai 11.4 and there were none.

mysql> select assessmentgradingid, SUBMITTEDDATE, PUBLISHEDANSWERID, RATIONALE, ANSWERTEXT, AUTOSCORE, OVERRIDESCORE, comments from sam_itemgrading_t where submitteddate > "2017-09-01 00:00:00" and submitteddate < "2017-12-01 00:00:00"and autoscore is null group by assessmentgradingid;
Empty set (3 min 7.57 sec)

also, since upgrading, there have been 35 instances of samigo assessments with null autoscores:

mysql> select count(*) from (select assessmentgradingid from sam_itemgrading_t where submitteddate > "2018-01-01 00:00:00" and autoscore is null group by assessmentgradingid) as t;
+----------+
| count(*) |
+----------+
|       34 |
+----------+
1 row in set (6 min 6.16 sec)


Then checking our tomcat logs, these errors that happened in the minutes after the bad scoring, I'm seeing these errors:

04-Feb-2018 21:37:12.495 ERROR [ajp-bio-8009-exec-11] org.sakaiproject.tool.assessment.services.GradingService.storeGrades unable to retrive itemDataIfc for: null
04-Feb-2018 21:37:12.495 ERROR [ajp-bio-8009-exec-11] org.sakaiproject.tool.assessment.services.GradingService.storeGrades unable to retrive itemDataIfc for: null


unfortunately, I still don't know how to reproduce the problem, but if those errors in the logs are related, searching the code base for those errors, I see:

GradingService.java
(line 937)
        itemId = itemGrading.getPublishedItemId();
        ItemDataIfc item = (ItemDataIfc) publishedItemHash.get(itemId);
        if (item == null) {
        //this probably shouldn't happen
        log.error("unable to retrive itemDataIfc for: {}", publishedItemHash.get(itemId));
        continue;
        }

(line 1048)
      //SAM-1724 it's possible the item is not in the hash -DH
        if (item == null) {
        log.error("unable to retrive itemDataIfc for: " + publishedItemHash.get(itemId));
        continue;
        }


If this indeed the problem, what would cause the item to not be in the hash?

Thanks,

Austin





On Wed, Feb 7, 2018 at 12:53 PM, John Ansorge <[hidden email]> wrote:

I made the SAM-3191 patch, although it’s been a while so my memory is rather hazy.  I’m pretty sure that what you’re describing is a different bug since it seems to affect “Multiple Choice, Single Selection” questions.  If I recall correctly the change in SAM-3191 is within a block of code only run for “Multiple Choice, Multiple Correct” questions so it would not do anything to fix the other kinds of multiple choice questions. 

 

But it is certainly possible there’s another point in the code where two floats are being compared with too much precision for MC,SS questions, leading to a similar symptom. 

 

John

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Sanghyun Grace Jeon
Sent: Wednesday, February 07, 2018 1:05 PM
To: 'Austin' <[hidden email]>; sakai-dev <[hidden email]>
Subject: RE: [sakai-dev] Re: Samigo autoscore null for some answers

 

Dear Austin,

 

We are using Sakai 11.2 and SAM-3191 issue bit us this week. Student received 0, although s/he selected all correct answers. I just checked our database again, but I don’t see any null value. All 0 is recorded for totalautoscore, totaloverridescore and finalscore. We are using Mysql 5.5.56

 

S

 

From: [hidden email] [[hidden email]] On Behalf Of Austin
Sent: Wednesday, February 7, 2018 12:49 PM
To: sakai-dev <[hidden email]>
Subject: [sakai-dev] Re: Samigo autoscore null for some answers

 

Hello All,

 

Just wondering if anyone has any idea about this problem?  (btw, we're using Sakai 11.4)

 

I was thinking that this could be due to https://jira.sakaiproject.org/browse/SAM-3191 and maybe it could be, but the fact that the autoscore is null for that one question (as well as null overridescore for two questions), kind of bothers me... what would cause them to be null in the first place?

 

Thanks,

 

Austin

 

On Wed, Jan 24, 2018 at 12:11 PM, Austin <[hidden email]> wrote:

Hello Sakai Devs,

 

Has anyone ever seen anything in Samigo (tests & quizzes) where after a student submits their answers some of their answers' autoscore remain NULL?  Even though they answered correctly?

 

When this happens, the instructor's UI shows that they got the question correct but the score is 0 (which I assume is because of the NULL autoscore value)

 

I didn't see anything in JIRA that sounded similar (though maybe I wasn't searching the right keywords)

 

for example, here's what it looks like in the database.  All of the questions except the last one have an autoscore.  what's also strange is that the 2nd to the last one has a NULL overridescore.

 

mysql> select SUBMITTEDDATE, PUBLISHEDANSWERID, RATIONALE, ANSWERTEXT, AUTOSCORE, OVERRIDESCORE, comments from sam_itemgrading_t where assessmentgradingid = '2364664' order by submitteddate;

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

| SUBMITTEDDATE       | PUBLISHEDANSWERID | RATIONALE | ANSWERTEXT | AUTOSCORE | OVERRIDESCORE | comments |

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

| 2018-01-12 08:55:16 |          12318602 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 08:58:01 |          12318625 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 08:58:05 |          12318631 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:22 |          12318576 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:25 |          12318618 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:35 |          12318585 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:00 |          12318590 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:25 |          12318634 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:45 |          12318627 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:02:17 |          12318641 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:02:41 |          12318614 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:03:27 |          12318637 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:05:33 |          12318594 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:06:28 |          12318612 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:07:04 |          12318620 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:08:48 |          12318600 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:09:14 |          12318572 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:09:50 |          12318580 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:12:52 |          12318608 | NULL      | NULL       |         1 |          NULL | NULL     |

| 2018-01-12 09:15:06 |          12318644 |           | NULL       |      NULL |          NULL | NULL     |

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

20 rows in set (0.01 sec)

 

 

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.


--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.



--
Thanks:
  David Hutchins, KK4LHO

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.
Austin Nakasone Austin Nakasone
Reply | Threaded
Open this post in threaded view
|

Re: [sakai-dev] Re: Samigo autoscore null for some answers

Hello David,

Thanks for replying.  We also thought that there might be a correlation with the instructor updating settings while the students were taking the exam because one of the instructors mentioned they might have been.  But checking the instructors activity, they didn't update test settings until 30 minutes after the student submitted the test.  Then two days later a different student taking the same exam experienced the same problem, but this time the instructor didn't edit anything.

I also checked 3 other instances of the issue (all separate sites / tests) and two out of the three didn't have any instructor activity at the time the student was taking the test, nor did they edit any settings.  In the third test submission, the instructor happened to be adding tons of content to resources/dropbox (perhaps through webdav?), but I wouldn't think that that was related to mis-scoring samigo?

I suppose I can add the itemId to the log message, but I'm not sure how much that will help because I think the real problem is that the item is not getting into the publishedItemHash

In an normal successfully case, I tried debugging the spot that logs the message "unable to retrive itemDataIfc for: null", and trying to trace back to where the publishedItemHash "could" have missed getting the item added to it, I got as far as ContextUtil.java lookupBean(String beanName), but then it goes off into JSF where I'm not familiar with debugging.  So I'm guessing that somewhere in JSF, the submitted test answer is not getting added to the hash.  And Neal C. also commented about this in SAM-1724, "According to David H. (David Horwitz?) - 'Its difficult - these are errors seen in production environments and the path to reproduce them is often obscure - often related to JSF state ...'"

Unfortunately, I still don't know how to reproduce the issue, but I also think it's pretty strange that checking our DB, I don't see any instances of this when were on 10.7, but now we're seeing it since upgrading to 11.4

Thanks,

Austin

On Wed, Feb 14, 2018 at 4:04 AM, David Hutchins <[hidden email]> wrote:
I posted to this list a few weeks back, in regards to similar issues.

[sakai-dev] SAM-1724 still a production issue.
>
> Greetings Devs,
>   We have encountered this issue covered up in SAM-1724 a few times recently. The work done in that issue effectively prevents any exception being thrown, and generates a fairly useless log message. The issue as we have seen it present itself is that > the grade in Samigo will be fine, but a 0 will get pushed to the gradebook. Of course there is no indication to the student or instructor that anything has gone wrong, other than the 0 grade in the gradebook, with the message "unable to retrive itemDataIfc > for: null" being logged. Due to the way the log message is implemented it always logs "null" instead of the ItemId. I assume that there is some sort of session state corruption at play, but as of yet have been unable to reproduce this in a dev environment. 
> My question the the group is if anyone has any way to reproduce, or other ideas that could help get to the core of this issue and actually fix it, rather than just covering it up?
> Thanks:
>    -David

The logging is broken for sure, but I would be curious to find a solid way to reproduce, and maybe understand a bit better how the issue is encountered. In our production logs, there does seem to be a correlation with the instructor working in the grading screen (in samigo, not GB) while the student submits. I have not been able to confirm if that is a cause, or just coincidental though. 

On Tue, Feb 13, 2018 at 2:37 PM, Austin <[hidden email]> wrote:
I wonder if this is a new Samigo bug introduced in Sakai 11?

I checked our DB for assessments with null 'autoscore' prior to us upgrading to Sakai 11.4 and there were none.

mysql> select assessmentgradingid, SUBMITTEDDATE, PUBLISHEDANSWERID, RATIONALE, ANSWERTEXT, AUTOSCORE, OVERRIDESCORE, comments from sam_itemgrading_t where submitteddate > "2017-09-01 00:00:00" and submitteddate < "2017-12-01 00:00:00"and autoscore is null group by assessmentgradingid;
Empty set (3 min 7.57 sec)

also, since upgrading, there have been 35 instances of samigo assessments with null autoscores:

mysql> select count(*) from (select assessmentgradingid from sam_itemgrading_t where submitteddate > "2018-01-01 00:00:00" and autoscore is null group by assessmentgradingid) as t;
+----------+
| count(*) |
+----------+
|       34 |
+----------+
1 row in set (6 min 6.16 sec)


Then checking our tomcat logs, these errors that happened in the minutes after the bad scoring, I'm seeing these errors:

04-Feb-2018 21:37:12.495 ERROR [ajp-bio-8009-exec-11] org.sakaiproject.tool.assessment.services.GradingService.storeGrades unable to retrive itemDataIfc for: null
04-Feb-2018 21:37:12.495 ERROR [ajp-bio-8009-exec-11] org.sakaiproject.tool.assessment.services.GradingService.storeGrades unable to retrive itemDataIfc for: null


unfortunately, I still don't know how to reproduce the problem, but if those errors in the logs are related, searching the code base for those errors, I see:

GradingService.java
(line 937)
        itemId = itemGrading.getPublishedItemId();
        ItemDataIfc item = (ItemDataIfc) publishedItemHash.get(itemId);
        if (item == null) {
        //this probably shouldn't happen
        log.error("unable to retrive itemDataIfc for: {}", publishedItemHash.get(itemId));
        continue;
        }

(line 1048)
      //SAM-1724 it's possible the item is not in the hash -DH
        if (item == null) {
        log.error("unable to retrive itemDataIfc for: " + publishedItemHash.get(itemId));
        continue;
        }


If this indeed the problem, what would cause the item to not be in the hash?

Thanks,

Austin





On Wed, Feb 7, 2018 at 12:53 PM, John Ansorge <[hidden email]> wrote:

I made the SAM-3191 patch, although it’s been a while so my memory is rather hazy.  I’m pretty sure that what you’re describing is a different bug since it seems to affect “Multiple Choice, Single Selection” questions.  If I recall correctly the change in SAM-3191 is within a block of code only run for “Multiple Choice, Multiple Correct” questions so it would not do anything to fix the other kinds of multiple choice questions. 

 

But it is certainly possible there’s another point in the code where two floats are being compared with too much precision for MC,SS questions, leading to a similar symptom. 

 

John

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Sanghyun Grace Jeon
Sent: Wednesday, February 07, 2018 1:05 PM
To: 'Austin' <[hidden email]>; sakai-dev <[hidden email]>
Subject: RE: [sakai-dev] Re: Samigo autoscore null for some answers

 

Dear Austin,

 

We are using Sakai 11.2 and SAM-3191 issue bit us this week. Student received 0, although s/he selected all correct answers. I just checked our database again, but I don’t see any null value. All 0 is recorded for totalautoscore, totaloverridescore and finalscore. We are using Mysql 5.5.56

 

S

 

From: [hidden email] [[hidden email]] On Behalf Of Austin
Sent: Wednesday, February 7, 2018 12:49 PM
To: sakai-dev <[hidden email]>
Subject: [sakai-dev] Re: Samigo autoscore null for some answers

 

Hello All,

 

Just wondering if anyone has any idea about this problem?  (btw, we're using Sakai 11.4)

 

I was thinking that this could be due to https://jira.sakaiproject.org/browse/SAM-3191 and maybe it could be, but the fact that the autoscore is null for that one question (as well as null overridescore for two questions), kind of bothers me... what would cause them to be null in the first place?

 

Thanks,

 

Austin

 

On Wed, Jan 24, 2018 at 12:11 PM, Austin <[hidden email]> wrote:

Hello Sakai Devs,

 

Has anyone ever seen anything in Samigo (tests & quizzes) where after a student submits their answers some of their answers' autoscore remain NULL?  Even though they answered correctly?

 

When this happens, the instructor's UI shows that they got the question correct but the score is 0 (which I assume is because of the NULL autoscore value)

 

I didn't see anything in JIRA that sounded similar (though maybe I wasn't searching the right keywords)

 

for example, here's what it looks like in the database.  All of the questions except the last one have an autoscore.  what's also strange is that the 2nd to the last one has a NULL overridescore.

 

mysql> select SUBMITTEDDATE, PUBLISHEDANSWERID, RATIONALE, ANSWERTEXT, AUTOSCORE, OVERRIDESCORE, comments from sam_itemgrading_t where assessmentgradingid = '2364664' order by submitteddate;

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

| SUBMITTEDDATE       | PUBLISHEDANSWERID | RATIONALE | ANSWERTEXT | AUTOSCORE | OVERRIDESCORE | comments |

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

| 2018-01-12 08:55:16 |          12318602 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 08:58:01 |          12318625 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 08:58:05 |          12318631 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:22 |          12318576 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:25 |          12318618 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:35 |          12318585 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:00 |          12318590 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:25 |          12318634 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:45 |          12318627 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:02:17 |          12318641 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:02:41 |          12318614 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:03:27 |          12318637 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:05:33 |          12318594 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:06:28 |          12318612 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:07:04 |          12318620 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:08:48 |          12318600 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:09:14 |          12318572 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:09:50 |          12318580 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:12:52 |          12318608 | NULL      | NULL       |         1 |          NULL | NULL     |

| 2018-01-12 09:15:06 |          12318644 |           | NULL       |      NULL |          NULL | NULL     |

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

20 rows in set (0.01 sec)

 

 

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.


--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.



--
Thanks:
  David Hutchins, KK4LHO

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.
Austin Nakasone Austin Nakasone
Reply | Threaded
Open this post in threaded view
|

Re: [sakai-dev] Re: Samigo autoscore null for some answers

I've started a new JIRA for this:  https://jira.sakaiproject.org/browse/SAM-3368

Also, we found that this happened a few more times since last week and one of them had to do with a "matching" question, so it's not limited "multiple choice single correct" as previously thought.
We also asked one of the students affected if they noticed anything strange while taking the test and she mentioned that the site was slow and she had to reload multiple times... I wonder if she somehow double submitted the test or an answer and that confused the samigo grading?

Also, David H.  since you saw this or a similar issue, would you be able to run this query and see if it returns anything?  (WARNING this could be a very very long running query if you have a lot of data in sam_itemgrading_t, e.g. we have 40million+ rows)

select assessmentgradingid,submitteddate from sam_itemgrading_t where autoscore is null group by assessmentgradingid;

Thanks,

Austin

On Wed, Feb 14, 2018 at 10:55 AM, Austin <[hidden email]> wrote:
Hello David,

Thanks for replying.  We also thought that there might be a correlation with the instructor updating settings while the students were taking the exam because one of the instructors mentioned they might have been.  But checking the instructors activity, they didn't update test settings until 30 minutes after the student submitted the test.  Then two days later a different student taking the same exam experienced the same problem, but this time the instructor didn't edit anything.

I also checked 3 other instances of the issue (all separate sites / tests) and two out of the three didn't have any instructor activity at the time the student was taking the test, nor did they edit any settings.  In the third test submission, the instructor happened to be adding tons of content to resources/dropbox (perhaps through webdav?), but I wouldn't think that that was related to mis-scoring samigo?

I suppose I can add the itemId to the log message, but I'm not sure how much that will help because I think the real problem is that the item is not getting into the publishedItemHash

In an normal successfully case, I tried debugging the spot that logs the message "unable to retrive itemDataIfc for: null", and trying to trace back to where the publishedItemHash "could" have missed getting the item added to it, I got as far as ContextUtil.java lookupBean(String beanName), but then it goes off into JSF where I'm not familiar with debugging.  So I'm guessing that somewhere in JSF, the submitted test answer is not getting added to the hash.  And Neal C. also commented about this in SAM-1724, "According to David H. (David Horwitz?) - 'Its difficult - these are errors seen in production environments and the path to reproduce them is often obscure - often related to JSF state ...'"

Unfortunately, I still don't know how to reproduce the issue, but I also think it's pretty strange that checking our DB, I don't see any instances of this when were on 10.7, but now we're seeing it since upgrading to 11.4

Thanks,

Austin

On Wed, Feb 14, 2018 at 4:04 AM, David Hutchins <[hidden email]> wrote:
I posted to this list a few weeks back, in regards to similar issues.

[sakai-dev] SAM-1724 still a production issue.
>
> Greetings Devs,
>   We have encountered this issue covered up in SAM-1724 a few times recently. The work done in that issue effectively prevents any exception being thrown, and generates a fairly useless log message. The issue as we have seen it present itself is that > the grade in Samigo will be fine, but a 0 will get pushed to the gradebook. Of course there is no indication to the student or instructor that anything has gone wrong, other than the 0 grade in the gradebook, with the message "unable to retrive itemDataIfc > for: null" being logged. Due to the way the log message is implemented it always logs "null" instead of the ItemId. I assume that there is some sort of session state corruption at play, but as of yet have been unable to reproduce this in a dev environment. 
> My question the the group is if anyone has any way to reproduce, or other ideas that could help get to the core of this issue and actually fix it, rather than just covering it up?
> Thanks:
>    -David

The logging is broken for sure, but I would be curious to find a solid way to reproduce, and maybe understand a bit better how the issue is encountered. In our production logs, there does seem to be a correlation with the instructor working in the grading screen (in samigo, not GB) while the student submits. I have not been able to confirm if that is a cause, or just coincidental though. 

On Tue, Feb 13, 2018 at 2:37 PM, Austin <[hidden email]> wrote:
I wonder if this is a new Samigo bug introduced in Sakai 11?

I checked our DB for assessments with null 'autoscore' prior to us upgrading to Sakai 11.4 and there were none.

mysql> select assessmentgradingid, SUBMITTEDDATE, PUBLISHEDANSWERID, RATIONALE, ANSWERTEXT, AUTOSCORE, OVERRIDESCORE, comments from sam_itemgrading_t where submitteddate > "2017-09-01 00:00:00" and submitteddate < "2017-12-01 00:00:00"and autoscore is null group by assessmentgradingid;
Empty set (3 min 7.57 sec)

also, since upgrading, there have been 35 instances of samigo assessments with null autoscores:

mysql> select count(*) from (select assessmentgradingid from sam_itemgrading_t where submitteddate > "2018-01-01 00:00:00" and autoscore is null group by assessmentgradingid) as t;
+----------+
| count(*) |
+----------+
|       34 |
+----------+
1 row in set (6 min 6.16 sec)


Then checking our tomcat logs, these errors that happened in the minutes after the bad scoring, I'm seeing these errors:

04-Feb-2018 21:37:12.495 ERROR [ajp-bio-8009-exec-11] org.sakaiproject.tool.assessment.services.GradingService.storeGrades unable to retrive itemDataIfc for: null
04-Feb-2018 21:37:12.495 ERROR [ajp-bio-8009-exec-11] org.sakaiproject.tool.assessment.services.GradingService.storeGrades unable to retrive itemDataIfc for: null


unfortunately, I still don't know how to reproduce the problem, but if those errors in the logs are related, searching the code base for those errors, I see:

GradingService.java
(line 937)
        itemId = itemGrading.getPublishedItemId();
        ItemDataIfc item = (ItemDataIfc) publishedItemHash.get(itemId);
        if (item == null) {
        //this probably shouldn't happen
        log.error("unable to retrive itemDataIfc for: {}", publishedItemHash.get(itemId));
        continue;
        }

(line 1048)
      //SAM-1724 it's possible the item is not in the hash -DH
        if (item == null) {
        log.error("unable to retrive itemDataIfc for: " + publishedItemHash.get(itemId));
        continue;
        }


If this indeed the problem, what would cause the item to not be in the hash?

Thanks,

Austin





On Wed, Feb 7, 2018 at 12:53 PM, John Ansorge <[hidden email]> wrote:

I made the SAM-3191 patch, although it’s been a while so my memory is rather hazy.  I’m pretty sure that what you’re describing is a different bug since it seems to affect “Multiple Choice, Single Selection” questions.  If I recall correctly the change in SAM-3191 is within a block of code only run for “Multiple Choice, Multiple Correct” questions so it would not do anything to fix the other kinds of multiple choice questions. 

 

But it is certainly possible there’s another point in the code where two floats are being compared with too much precision for MC,SS questions, leading to a similar symptom. 

 

John

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Sanghyun Grace Jeon
Sent: Wednesday, February 07, 2018 1:05 PM
To: 'Austin' <[hidden email]>; sakai-dev <[hidden email]>
Subject: RE: [sakai-dev] Re: Samigo autoscore null for some answers

 

Dear Austin,

 

We are using Sakai 11.2 and SAM-3191 issue bit us this week. Student received 0, although s/he selected all correct answers. I just checked our database again, but I don’t see any null value. All 0 is recorded for totalautoscore, totaloverridescore and finalscore. We are using Mysql 5.5.56

 

S

 

From: [hidden email] [[hidden email]] On Behalf Of Austin
Sent: Wednesday, February 7, 2018 12:49 PM
To: sakai-dev <[hidden email]>
Subject: [sakai-dev] Re: Samigo autoscore null for some answers

 

Hello All,

 

Just wondering if anyone has any idea about this problem?  (btw, we're using Sakai 11.4)

 

I was thinking that this could be due to https://jira.sakaiproject.org/browse/SAM-3191 and maybe it could be, but the fact that the autoscore is null for that one question (as well as null overridescore for two questions), kind of bothers me... what would cause them to be null in the first place?

 

Thanks,

 

Austin

 

On Wed, Jan 24, 2018 at 12:11 PM, Austin <[hidden email]> wrote:

Hello Sakai Devs,

 

Has anyone ever seen anything in Samigo (tests & quizzes) where after a student submits their answers some of their answers' autoscore remain NULL?  Even though they answered correctly?

 

When this happens, the instructor's UI shows that they got the question correct but the score is 0 (which I assume is because of the NULL autoscore value)

 

I didn't see anything in JIRA that sounded similar (though maybe I wasn't searching the right keywords)

 

for example, here's what it looks like in the database.  All of the questions except the last one have an autoscore.  what's also strange is that the 2nd to the last one has a NULL overridescore.

 

mysql> select SUBMITTEDDATE, PUBLISHEDANSWERID, RATIONALE, ANSWERTEXT, AUTOSCORE, OVERRIDESCORE, comments from sam_itemgrading_t where assessmentgradingid = '2364664' order by submitteddate;

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

| SUBMITTEDDATE       | PUBLISHEDANSWERID | RATIONALE | ANSWERTEXT | AUTOSCORE | OVERRIDESCORE | comments |

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

| 2018-01-12 08:55:16 |          12318602 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 08:58:01 |          12318625 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 08:58:05 |          12318631 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:22 |          12318576 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:25 |          12318618 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:00:35 |          12318585 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:00 |          12318590 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:25 |          12318634 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:01:45 |          12318627 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:02:17 |          12318641 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:02:41 |          12318614 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:03:27 |          12318637 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:05:33 |          12318594 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:06:28 |          12318612 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:07:04 |          12318620 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:08:48 |          12318600 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:09:14 |          12318572 |           | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:09:50 |          12318580 | NULL      | NULL       |         1 |             0 | NULL     |

| 2018-01-12 09:12:52 |          12318608 | NULL      | NULL       |         1 |          NULL | NULL     |

| 2018-01-12 09:15:06 |          12318644 |           | NULL       |      NULL |          NULL | NULL     |

+---------------------+-------------------+-----------+------------+-----------+---------------+----------+

20 rows in set (0.01 sec)

 

 

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.


--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.



--
Thanks:
  David Hutchins, KK4LHO

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.


--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/a/apereo.org/group/sakai-dev/.