[sakai2-tcc] Summary of Tools Survey comments

classic Classic list List threaded Threaded
5 messages Options
Neal Caidin-3 Neal Caidin-3
Reply | Threaded
Open this post in threaded view
|

[sakai2-tcc] Summary of Tools Survey comments

Hi TCC,

I found summarizing the Survey comments challenging. I hope it proves valuable.

Here's what I've got so far:


Here are my questions about it:

1) Is it "good enough"?  I think it is, though not sure about that arbitrary column I added called "Calculation". More about that in a minute

2) This probably needs a Legend added. You'll notice I have a Positive - Kudos column, but no Negative column. This is because anything that could be perceived as negative, I tried to categorize more specifically what the feedback is.

Legend:

Development resources - we did not ask for volunteers, but one institution said that if other institutions were interested in putting effort behind Gradebook (Gradebook 1), they might throw in development resources too. I thought that was worthy of it's own column.

Performance concerns - broadly speaking. This includes not only performance under load, but also if the tool does not seem to be meeting institutional requirements.
Alternative Tool preference - for example, using Gradebook2 instead of Gradebook. Or using an external Syllabus tool instead of Sakai Syllabus, or even just posting Word/PDF docs instead of using tool at all.

Request for Feature or Bug fix - sometimes specific requests, sometimes expressed as a general desire to see the features enhanced. 

Information Only - not positive, not negative, just informational. A large percentage of these turned out to be mentioning plans to try out/pilot a tool. Made me wonder if it warrants a separate column for "Planning to Pilot"?  I would have to go through the data again, but might be worth it.

High Use - comment specifically said that the tool was highly used

Low Use - comment specifically said that the tool is used sparsely 

calculation - I made up this formula to see if it might indicate the relative interest level in the tool. = Total comments for a tool - Low use - Alternative tools - Performance concerns = interest level in keeping the tool/ further development? See what you think. Not sure if we should keep this column. Was done as an intellectual exercise. For the most dramatic example of how this might normalize the data, see the Chat tool. Lots of comments, but mostly negative vs Lessons which has lots of comments but mostly positive. 

I know this a busy time, but if you have about 10 minutes to spare to look at the data and think about it, that would be cool (I know that is asking a lot right now.)

Thanks,
Neal



Neal Caidin
Sakai CLE Community Coordinator
Skype: nealkdin
Twitter: ncaidin










_______________________________________________
sakai2-tcc mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
Steve Swinsburg-3 Steve Swinsburg-3
Reply | Threaded
Open this post in threaded view
|

Re: [sakai2-tcc] Summary of Tools Survey comments

Hi Neal,

The numbers of comments are interesting but it would be good to get an anonymised list of the actual comments because some might be able to be addressed or filter out if the functionality already exists and the responder wasn't aware, for example.

Sent from my iPad

On 21/08/2013, at 5:17, Neal Caidin <[hidden email]> wrote:

Hi TCC,

I found summarizing the Survey comments challenging. I hope it proves valuable.

Here's what I've got so far:


Here are my questions about it:

1) Is it "good enough"?  I think it is, though not sure about that arbitrary column I added called "Calculation". More about that in a minute

2) This probably needs a Legend added. You'll notice I have a Positive - Kudos column, but no Negative column. This is because anything that could be perceived as negative, I tried to categorize more specifically what the feedback is.

Legend:

Development resources - we did not ask for volunteers, but one institution said that if other institutions were interested in putting effort behind Gradebook (Gradebook 1), they might throw in development resources too. I thought that was worthy of it's own column.

Performance concerns - broadly speaking. This includes not only performance under load, but also if the tool does not seem to be meeting institutional requirements.
Alternative Tool preference - for example, using Gradebook2 instead of Gradebook. Or using an external Syllabus tool instead of Sakai Syllabus, or even just posting Word/PDF docs instead of using tool at all.

Request for Feature or Bug fix - sometimes specific requests, sometimes expressed as a general desire to see the features enhanced. 

Information Only - not positive, not negative, just informational. A large percentage of these turned out to be mentioning plans to try out/pilot a tool. Made me wonder if it warrants a separate column for "Planning to Pilot"?  I would have to go through the data again, but might be worth it.

High Use - comment specifically said that the tool was highly used

Low Use - comment specifically said that the tool is used sparsely 

calculation - I made up this formula to see if it might indicate the relative interest level in the tool. = Total comments for a tool - Low use - Alternative tools - Performance concerns = interest level in keeping the tool/ further development? See what you think. Not sure if we should keep this column. Was done as an intellectual exercise. For the most dramatic example of how this might normalize the data, see the Chat tool. Lots of comments, but mostly negative vs Lessons which has lots of comments but mostly positive. 

I know this a busy time, but if you have about 10 minutes to spare to look at the data and think about it, that would be cool (I know that is asking a lot right now.)

Thanks,
Neal



Neal Caidin
Sakai CLE Community Coordinator
Skype: nealkdin
Twitter: ncaidin









_______________________________________________
sakai2-tcc mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc

_______________________________________________
sakai2-tcc mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
Neal Caidin-3 Neal Caidin-3
Reply | Threaded
Open this post in threaded view
|

Re: [sakai2-tcc] Summary of Tools Survey comments

Hi Steve,

We do have the list of comments. They are not fully anonymised because some folks referenced their institution in the comment. It's on our shared Dropbox folder as Summary of Results Draft -> Summary_of_Comments.txt  .    Based on what I can see you should have access.

I could clean this up, remove references to institutions in the comments, and publish it with the other data. However there are one or two comments of this nature:
"Hate it!!  Tons of problems with this tool, not just bugs but terrible design."  - Would we want to publish something like this, even if the source is anonymous? It will still be associated with a tool, and it is not anonymous who works on tools. That's my only reluctance to publishing an anonymous list of comments.

If you (or anyone for that matter) sees a comment that is based on a misperception of a tool, please let me know and I'll be happy to follow up with the responding institution. 


Thanks,
Neal





Neal Caidin
Sakai CLE Community Coordinator
Skype: nealkdin
Twitter: ncaidin









On Aug 20, 2013, at 5:56 PM, Steve Swinsburg <[hidden email]> wrote:

Hi Neal,

The numbers of comments are interesting but it would be good to get an anonymised list of the actual comments because some might be able to be addressed or filter out if the functionality already exists and the responder wasn't aware, for example.

Sent from my iPad

On 21/08/2013, at 5:17, Neal Caidin <[hidden email]> wrote:

Hi TCC,

I found summarizing the Survey comments challenging. I hope it proves valuable.

Here's what I've got so far:


Here are my questions about it:

1) Is it "good enough"?  I think it is, though not sure about that arbitrary column I added called "Calculation". More about that in a minute

2) This probably needs a Legend added. You'll notice I have a Positive - Kudos column, but no Negative column. This is because anything that could be perceived as negative, I tried to categorize more specifically what the feedback is.

Legend:

Development resources - we did not ask for volunteers, but one institution said that if other institutions were interested in putting effort behind Gradebook (Gradebook 1), they might throw in development resources too. I thought that was worthy of it's own column.

Performance concerns - broadly speaking. This includes not only performance under load, but also if the tool does not seem to be meeting institutional requirements.
Alternative Tool preference - for example, using Gradebook2 instead of Gradebook. Or using an external Syllabus tool instead of Sakai Syllabus, or even just posting Word/PDF docs instead of using tool at all.

Request for Feature or Bug fix - sometimes specific requests, sometimes expressed as a general desire to see the features enhanced. 

Information Only - not positive, not negative, just informational. A large percentage of these turned out to be mentioning plans to try out/pilot a tool. Made me wonder if it warrants a separate column for "Planning to Pilot"?  I would have to go through the data again, but might be worth it.

High Use - comment specifically said that the tool was highly used

Low Use - comment specifically said that the tool is used sparsely 

calculation - I made up this formula to see if it might indicate the relative interest level in the tool. = Total comments for a tool - Low use - Alternative tools - Performance concerns = interest level in keeping the tool/ further development? See what you think. Not sure if we should keep this column. Was done as an intellectual exercise. For the most dramatic example of how this might normalize the data, see the Chat tool. Lots of comments, but mostly negative vs Lessons which has lots of comments but mostly positive. 

I know this a busy time, but if you have about 10 minutes to spare to look at the data and think about it, that would be cool (I know that is asking a lot right now.)

Thanks,
Neal



Neal Caidin
Sakai CLE Community Coordinator
Skype: nealkdin
Twitter: ncaidin









_______________________________________________
sakai2-tcc mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc


_______________________________________________
sakai2-tcc mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
markjnorton markjnorton
Reply | Threaded
Open this post in threaded view
|

Re: [sakai2-tcc] Summary of Tools Survey comments

In reply to this post by Neal Caidin-3
Some comments:
  1. Give the document a better name than "Comments Summary".
  2. What is the "Calculation" column?
  3. Find a way to add your Legend descriptions to the document.
- Mark

On 8/20/2013 3:17 PM, Neal Caidin wrote:
Hi TCC,

I found summarizing the Survey comments challenging. I hope it proves valuable.

Here's what I've got so far:


Here are my questions about it:

1) Is it "good enough"?  I think it is, though not sure about that arbitrary column I added called "Calculation". More about that in a minute

2) This probably needs a Legend added. You'll notice I have a Positive - Kudos column, but no Negative column. This is because anything that could be perceived as negative, I tried to categorize more specifically what the feedback is.

Legend:

Development resources - we did not ask for volunteers, but one institution said that if other institutions were interested in putting effort behind Gradebook (Gradebook 1), they might throw in development resources too. I thought that was worthy of it's own column.

Performance concerns - broadly speaking. This includes not only performance under load, but also if the tool does not seem to be meeting institutional requirements.
Alternative Tool preference - for example, using Gradebook2 instead of Gradebook. Or using an external Syllabus tool instead of Sakai Syllabus, or even just posting Word/PDF docs instead of using tool at all.

Request for Feature or Bug fix - sometimes specific requests, sometimes expressed as a general desire to see the features enhanced. 

Information Only - not positive, not negative, just informational. A large percentage of these turned out to be mentioning plans to try out/pilot a tool. Made me wonder if it warrants a separate column for "Planning to Pilot"?  I would have to go through the data again, but might be worth it.

High Use - comment specifically said that the tool was highly used

Low Use - comment specifically said that the tool is used sparsely 

calculation - I made up this formula to see if it might indicate the relative interest level in the tool. = Total comments for a tool - Low use - Alternative tools - Performance concerns = interest level in keeping the tool/ further development? See what you think. Not sure if we should keep this column. Was done as an intellectual exercise. For the most dramatic example of how this might normalize the data, see the Chat tool. Lots of comments, but mostly negative vs Lessons which has lots of comments but mostly positive. 

I know this a busy time, but if you have about 10 minutes to spare to look at the data and think about it, that would be cool (I know that is asking a lot right now.)

Thanks,
Neal



Neal Caidin
Sakai CLE Community Coordinator
Skype: nealkdin
Twitter: ncaidin











_______________________________________________
sakai2-tcc mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc


_______________________________________________
sakai2-tcc mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc
Neal Caidin-3 Neal Caidin-3
Reply | Threaded
Open this post in threaded view
|

Re: [sakai2-tcc] Summary of Tools Survey comments

Hi Mark,

1. I agree. How about "Comments Metrics" ?

2. It is a silly intellectual exercise I am experimenting with. I'll probably delete it before broadly publishing.  I was trying to find a way to make some inferences about the number and types of comments, which is dangerous with limited data (i.e. easy to make incorrect inferences about the meaning of the data).  Could the reason some tools received less comments be because folks are basically content with those tools, and there is no excitement/need around further development? Could the reason other tools received more comments be because people are either unhappy with the tool or because they are excited about it and want to see more development?  My assumption is that Low use, Alternative tools, and Performance concerns are indications of unhappiness with a tool whereas all the other columns representing interest in a tool or that the tool is doing a good job meeting needs. 

Fwiw, I kind of think it is pretty cool that we got so many comments when it was an optional field. 

3. I agree and will add the Legend.

Thanks,
Neal



Neal Caidin
Sakai CLE Community Coordinator
Skype: nealkdin
Twitter: ncaidin









On Aug 21, 2013, at 8:53 AM, "Mark J. Norton" <[hidden email]> wrote:

Some comments:
  1. Give the document a better name than "Comments Summary".
  2. What is the "Calculation" column?
  3. Find a way to add your Legend descriptions to the document.
- Mark

On 8/20/2013 3:17 PM, Neal Caidin wrote:
Hi TCC,

I found summarizing the Survey comments challenging. I hope it proves valuable.

Here's what I've got so far:


Here are my questions about it:

1) Is it "good enough"?  I think it is, though not sure about that arbitrary column I added called "Calculation". More about that in a minute

2) This probably needs a Legend added. You'll notice I have a Positive - Kudos column, but no Negative column. This is because anything that could be perceived as negative, I tried to categorize more specifically what the feedback is.

Legend:

Development resources - we did not ask for volunteers, but one institution said that if other institutions were interested in putting effort behind Gradebook (Gradebook 1), they might throw in development resources too. I thought that was worthy of it's own column.

Performance concerns - broadly speaking. This includes not only performance under load, but also if the tool does not seem to be meeting institutional requirements.
Alternative Tool preference - for example, using Gradebook2 instead of Gradebook. Or using an external Syllabus tool instead of Sakai Syllabus, or even just posting Word/PDF docs instead of using tool at all.

Request for Feature or Bug fix - sometimes specific requests, sometimes expressed as a general desire to see the features enhanced. 

Information Only - not positive, not negative, just informational. A large percentage of these turned out to be mentioning plans to try out/pilot a tool. Made me wonder if it warrants a separate column for "Planning to Pilot"?  I would have to go through the data again, but might be worth it.

High Use - comment specifically said that the tool was highly used

Low Use - comment specifically said that the tool is used sparsely 

calculation - I made up this formula to see if it might indicate the relative interest level in the tool. = Total comments for a tool - Low use - Alternative tools - Performance concerns = interest level in keeping the tool/ further development? See what you think. Not sure if we should keep this column. Was done as an intellectual exercise. For the most dramatic example of how this might normalize the data, see the Chat tool. Lots of comments, but mostly negative vs Lessons which has lots of comments but mostly positive. 

I know this a busy time, but if you have about 10 minutes to spare to look at the data and think about it, that would be cool (I know that is asking a lot right now.)

Thanks,
Neal



Neal Caidin
Sakai CLE Community Coordinator
Skype: nealkdin
Twitter: ncaidin











_______________________________________________
sakai2-tcc mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc

_______________________________________________
sakai2-tcc mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc


_______________________________________________
sakai2-tcc mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/sakai2-tcc