Deprecating reports and warehouse tools.

classic Classic list List threaded Threaded
7 messages Options
Berg, Alan Berg, Alan
Reply | Threaded
Open this post in threaded view
|

Deprecating reports and warehouse tools.

Deprecating reports and warehouse tools.

Hi Product council and Maintenance team members,

the MT are meeting Wednesday to discuss depreciation recommendations for the PC as part of the Sakai 2.7 life cycle. This is good as the MT are the experts who deal with operational reality on a daily basis.

As Interim QA director I have the professional role to signal issues clearly. The Reports tool does not work in the QA tag Sakai 2.7 beta 4 and that the warehouse tool also appears to be unsupported. After discussion with Noah and Beth I believe that the tools do not have sufficient community support and are not being used widely enough to justify continued QA  effort. I therefore strongly recommend moving the tools from Enterprise core status back into contrib before the beta 5 tag is released and thus giving QA enough time to verify that removal was safe before 2.7 is tagged.

Alan

Alan Berg
Interim QA Director - The Sakai Foundation

Senior Developer / Quality Assurance
Group Education and Research Services
Central Computer Services
University of Amsterdam

http://home.uva.nl/a.m.berg








_______________________________________________
management mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/management

TO UNSUBSCRIBE: send email to [hidden email] with a subject of "unsubscribe"
Clay Fenlason Clay Fenlason
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating reports and warehouse tools.

Copying the portfolio list on this, to cast the net for informed
functional opinions a little wider.

No comments from me for the moment, I just want to ask an extra
question or two. Can you explain more precisely what "does not work"
means for the Reports tool?

What dependencies are there between the Reports and Warehouse tools
and any other parts of osp? Or perhaps even outside the osp suite?

~Clay

On Tue, Mar 9, 2010 at 5:50 PM, Berg, Alan <[hidden email]> wrote:

> Hi Product council and Maintenance team members,
>
> the MT are meeting Wednesday to discuss depreciation recommendations for the
> PC as part of the Sakai 2.7 life cycle. This is good as the MT are the
> experts who deal with operational reality on a daily basis.
>
> As Interim QA director I have the professional role to signal issues
> clearly. The Reports tool does not work in the QA tag Sakai 2.7 beta 4 and
> that the warehouse tool also appears to be unsupported. After discussion
> with Noah and Beth I believe that the tools do not have sufficient community
> support and are not being used widely enough to justify continued QA
> effort. I therefore strongly recommend moving the tools from Enterprise core
> status back into contrib before the beta 5 tag is released and thus giving
> QA enough time to verify that removal was safe before 2.7 is tagged.
>
> Alan
>
> Alan Berg
> Interim QA Director - The Sakai Foundation
>
> Senior Developer / Quality Assurance
> Group Education and Research Services
> Central Computer Services
> University of Amsterdam
>
> http://home.uva.nl/a.m.berg
>
>
>
>
>
>
>
>
> _______________________________________________
> management mailing list
> [hidden email]
> http://collab.sakaiproject.org/mailman/listinfo/management
>
> TO UNSUBSCRIBE: send email to [hidden email]
> with a subject of "unsubscribe"
>
_______________________________________________
management mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/management

TO UNSUBSCRIBE: send email to [hidden email] with a subject of "unsubscribe"
Berg, Alan Berg, Alan
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating reports and warehouse tools.

RE: [Management] Deprecating reports and warehouse tools.

Hi Clay,

Good questions:

1) The report tool did not start up in beta 3 or beta 4, this was due to remedial repairs that broke the tool in a difficult to diagnose way.
2) The warehouse tool has the same community support level as the reports tool, which at best is low.
3) External dependencies are not known to me and if exist are dangerous.

I think this is a good juncture for the OSP experts to cast their opinions.

Alan

Alan Berg
Interim QA Director - The Sakai Foundation

Senior Developer / Quality Assurance
Group Education and Research Services
Central Computer Services
University of Amsterdam

http://home.uva.nl/a.m.berg


Alan

Alan Berg
Interim QA Director - The Sakai Foundation

Senior Developer / Quality Assurance
Group Education and Research Services
Central Computer Services
University of Amsterdam

http://home.uva.nl/a.m.berg




-----Original Message-----
From: [hidden email] on behalf of Clay Fenlason
Sent: Wed 3/10/2010 0:20
To: [hidden email]; [hidden email]; portfolio
Subject: Re: [Management] Deprecating reports and warehouse tools.

Copying the portfolio list on this, to cast the net for informed
functional opinions a little wider.

No comments from me for the moment, I just want to ask an extra
question or two. Can you explain more precisely what "does not work"
means for the Reports tool?

What dependencies are there between the Reports and Warehouse tools
and any other parts of osp? Or perhaps even outside the osp suite?

~Clay

On Tue, Mar 9, 2010 at 5:50 PM, Berg, Alan <[hidden email]> wrote:
> Hi Product council and Maintenance team members,
>
> the MT are meeting Wednesday to discuss depreciation recommendations for the
> PC as part of the Sakai 2.7 life cycle. This is good as the MT are the
> experts who deal with operational reality on a daily basis.
>
> As Interim QA director I have the professional role to signal issues
> clearly. The Reports tool does not work in the QA tag Sakai 2.7 beta 4 and
> that the warehouse tool also appears to be unsupported. After discussion
> with Noah and Beth I believe that the tools do not have sufficient community
> support and are not being used widely enough to justify continued QA
> effort. I therefore strongly recommend moving the tools from Enterprise core
> status back into contrib before the beta 5 tag is released and thus giving
> QA enough time to verify that removal was safe before 2.7 is tagged.
>
> Alan
>
> Alan Berg
> Interim QA Director - The Sakai Foundation
>
> Senior Developer / Quality Assurance
> Group Education and Research Services
> Central Computer Services
> University of Amsterdam
>
> http://home.uva.nl/a.m.berg
>
>
>
>
>
>
>
>
> _______________________________________________
> management mailing list
> [hidden email]
> http://collab.sakaiproject.org/mailman/listinfo/management
>
> TO UNSUBSCRIBE: send email to [hidden email]
> with a subject of "unsubscribe"
>



_______________________________________________
management mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/management

TO UNSUBSCRIBE: send email to [hidden email] with a subject of "unsubscribe"
Anthony Whyte Anthony Whyte
Reply | Threaded
Open this post in threaded view
|

Re: Deprecating reports and warehouse tools.

I recommend that this issue be surfaced on sakai-dev and the production lists.  The developer/production community should have a chance to comment on the proposal.

Cheers,

Anthony


Begin forwarded message:

From: Anthony Whyte <[hidden email]>
Date: March 10, 2010 11:24:03 AM GMT+02:00
To: Michael Korcuska <[hidden email]>
Cc: Alan Berg <[hidden email]>, Beth Kirschner <[hidden email]>, [hidden email], Noah Botimer <[hidden email]>
Subject: Re: Test plans for reports

I recommend that this issue be surfaced on sakai-dev so that others can have a chance to comment.  I also agree with MK's hesitation regarding the proposed criteria for deprecation/removal proposed by Alan; better to simply focus on the absence of community support rather than assert what is absolutely true but can't at this moment be backed up by procedural documentation.

As for the mechanics of deprecation I assume we would do the following:

1) roll back the reports fix.  As a courtesy make sure reports at least starts up in Tomcat.
2) relocate reports and warehouse to contrib (strictly speaking this not absolutely necessary at the present time; we could do it when we have time)
3) either
a.  stealth reports and warehouse in trunk/2.7.x (if the projects are moved we will have to repoint .externals to their contrib locations)
b.  excise reports and warehouse from trunk/2.7.x builds (i.e., remove them from .externals)

Today I have the day off until 5 pm (GMT+2) when I head in to UCT to join in on the MT call.  Reports/warehouse are on our agenda.

Cheers,

Anthony



On Mar 10, 2010, at 1:29 AM, Berg, Alan wrote:

Hi Clay,

Good questions:

1) The report tool did not start up in beta 3 or beta 4, this was due to remedial repairs that broke the tool in a difficult to diagnose way.
2) The warehouse tool has the same community support level as the reports tool, which at best is low.
3) External dependencies are not known to me and if exist are dangerous.

I think this is a good juncture for the OSP experts to cast their opinions.

Alan

Alan Berg
Interim QA Director - The Sakai Foundation

Senior Developer / Quality Assurance
Group Education and Research Services
Central Computer Services
University of Amsterdam

http://home.uva.nl/a.m.berg


Alan

Alan Berg
Interim QA Director - The Sakai Foundation

Senior Developer / Quality Assurance
Group Education and Research Services
Central Computer Services
University of Amsterdam

http://home.uva.nl/a.m.berg




-----Original Message-----
From: [hidden email] on behalf of Clay Fenlason
Sent: Wed 3/10/2010 0:20
To: [hidden email]; [hidden email]; portfolio
Subject: Re: [Management] Deprecating reports and warehouse tools.

Copying the portfolio list on this, to cast the net for informed
functional opinions a little wider.

No comments from me for the moment, I just want to ask an extra
question or two. Can you explain more precisely what "does not work"
means for the Reports tool?

What dependencies are there between the Reports and Warehouse tools
and any other parts of osp? Or perhaps even outside the osp suite?

~Clay

On Tue, Mar 9, 2010 at 5:50 PM, Berg, Alan <[hidden email]> wrote:
> Hi Product council and Maintenance team members,
>
> the MT are meeting Wednesday to discuss depreciation recommendations for the
> PC as part of the Sakai 2.7 life cycle. This is good as the MT are the
> experts who deal with operational reality on a daily basis.
>
> As Interim QA director I have the professional role to signal issues
> clearly. The Reports tool does not work in the QA tag Sakai 2.7 beta 4 and
> that the warehouse tool also appears to be unsupported. After discussion
> with Noah and Beth I believe that the tools do not have sufficient community
> support and are not being used widely enough to justify continued QA
> effort. I therefore strongly recommend moving the tools from Enterprise core
> status back into contrib before the beta 5 tag is released and thus giving
> QA enough time to verify that removal was safe before 2.7 is tagged.
>
> Alan
>
> Alan Berg
> Interim QA Director - The Sakai Foundation
>
> Senior Developer / Quality Assurance
> Group Education and Research Services
> Central Computer Services
> University of Amsterdam
>
> http://home.uva.nl/a.m.berg
>
>
>
>
>
>
>
>
> _______________________________________________
> management mailing list
> [hidden email]
> http://collab.sakaiproject.org/mailman/listinfo/management
>
> TO UNSUBSCRIBE: send email to [hidden email]
> with a subject of "unsubscribe"
>


_______________________________________________
management mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/management

TO UNSUBSCRIBE: send email to [hidden email] with a subject of "unsubscribe"


_______________________________________________
management mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/management

TO UNSUBSCRIBE: send email to [hidden email] with a subject of "unsubscribe"

smime.p7s (3K) Download Attachment
Berg, Alan Berg, Alan
Reply | Threaded
Open this post in threaded view
|

Deprecating reports and warehouse tools.

Deprecating reports and warehouse tools.

Hi Dev and production list,

I have  recommended to the Product Council and the Maintenance team that the reports and warehouse tools are demoted from Sakai 2.7 as Enterprise core tools and placed back into contrib. Please feel free to comment and make your views known.  The  original email sent is given below


> > Hi Product council and Maintenance team members,
> >
> > the MT are meeting Wednesday to discuss depreciation recommendations for the
> > PC as part of the Sakai 2.7 life cycle. This is good as the MT are the
> > experts who deal with operational reality on a daily basis.
> >
> > As Interim QA director I have the professional role to signal issues
> > clearly. The Reports tool does not work in the QA tag Sakai 2.7 beta 4 and
> > that the warehouse tool also appears to be unsupported. After discussion
> > with Noah and Beth I believe that the tools do not have sufficient community
> > support and are not being used widely enough to justify continued QA
> > effort. I therefore strongly recommend moving the tools from Enterprise core
> > status back into contrib before the beta 5 tag is released and thus giving
> > QA enough time to verify that removal was safe before 2.7 is tagged.



Alan

Alan Berg
Interim QA Director - The Sakai Foundation

Senior Developer / Quality Assurance
Group Education and Research Services
Central Computer Services
University of Amsterdam

http://home.uva.nl/a.m.berg


_______________________________________________
management mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/management

TO UNSUBSCRIBE: send email to [hidden email] with a subject of "unsubscribe"
Anthony Whyte Anthony Whyte
Reply | Threaded
Open this post in threaded view
|

Re: [Building Sakai] Deprecating reports and warehouse tools.

Reports has failed for me on startup for the last two tags.  See


Cheers,

Anth


On Mar 10, 2010, at 6:44 PM, Maurer, Christopher Wayne wrote:

We use the reports tool in production here at IU, but do not use the warehouse tool.
And I think your statement is incorrect about the Reports tool not working in the beta 4 tag of 2.7.  I am able to get to the tool on qa3-us.

Chris

On Mar 10, 2010, at 5:05 AM, Berg, Alan wrote:

Hi Dev and production list,

I have  recommended to the Product Council and the Maintenance team that the reports and warehouse tools are demoted from Sakai 2.7 as Enterprise core tools and placed back into contrib. Please feel free to comment and make your views known.  The  original email sent is given below


> > Hi Product council and Maintenance team members,
> >
> > the MT are meeting Wednesday to discuss depreciation recommendations for the
> > PC as part of the Sakai 2.7 life cycle. This is good as the MT are the
> > experts who deal with operational reality on a daily basis.
> >
> > As Interim QA director I have the professional role to signal issues
> > clearly. The Reports tool does not work in the QA tag Sakai 2.7 beta 4 and
> > that the warehouse tool also appears to be unsupported. After discussion
> > with Noah and Beth I believe that the tools do not have sufficient community
> > support and are not being used widely enough to justify continued QA
> > effort. I therefore strongly recommend moving the tools from Enterprise core
> > status back into contrib before the beta 5 tag is released and thus giving
> > QA enough time to verify that removal was safe before 2.7 is tagged.



Alan

Alan Berg
Interim QA Director - The Sakai Foundation

Senior Developer / Quality Assurance
Group Education and Research Services
Central Computer Services
University of Amsterdam

http://home.uva.nl/a.m.berg

<ATT00001..txt>



_______________________________________________
management mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/management

TO UNSUBSCRIBE: send email to [hidden email] with a subject of "unsubscribe"

smime.p7s (3K) Download Attachment
khomotso khomotso
Reply | Threaded
Open this post in threaded view
|

Re: [Portfolio] Deprecating reports and warehouse tools.

In reply to this post by Berg, Alan
Thanks, John, for the explanation. Very helpful.

~Clay

On Wed, Mar 10, 2010 at 11:30 PM, John Bush <[hidden email]> wrote:

> Having work on a few projects that have attempted to leverage this infrastructure I'll chime in with my experiences/thoughts.
>
> Typically its portfolio use cases, K-12, or complete online instruction use cases that wish to leverage reporting use cases.  The problematic issues in Sakai are that typically people what to report on things across sites, and generally Sakai's API's are always structured to operate within the site context.   While leveraging the api's to do so is in theory possible, in practice the looping necessary is too slow.  It also requires Java development in order to do so.  Generally, for data mining, business intelligence use cases a direct SQL type solution is typical.  That is where Sakai's use of xml meta data is problematic.
>
> The warehouse was created to attempt to solve these issues by pulling the data out of sakai into a denormalized tables that are easier to report against.  The warehouse is a complicated home grown way of producing what people typically use views for.  The warehouse was also intended to shield reports from SQL changes that might occur from release to release.
>
> The reports tool on the other hand was an attempt to create a reporting infrastructure to make reports within Sakai that could have some level of parameterization and sharing capabilities.  Depending on what you want to report on warehousing may or may not be necessary.  That's probably why IU can use reports independently of the warehouse, or they have another warehouse solution.
>
> In the end, running the warehouse from within Sakai has been problematic for a number of reason.  The use of thread local and caching mechanisms within the system don't typically expect complete calls to every bit of data and out of memory issues and such can occur.  The warehousing on a very large system can take several hours if not half a day.   The complexity of the Spring configuration is a big learning curve for anyone attempting to build upon it, and there isn't much documentation to help.
>
> In a nutshell having worked with and built upon these tools, we are looking for other solutions, so I think the product council recommendation makes sense.  When you think about the issue more generally you end up looking at business intelligence and warehousing products: think pentaho, cognios, crystal reports, whatever...
>
> While ultimately it would be nice if more of the xml could move out into the database, I don't suspect anyone to sign up for that sort of work.  What we are looking at is just using views for this.   Both mysql and oracle have ways of parsing the xml directly, if you look at the assignment upgrade sql for 2.6 or 2.6.1 (I forget which one) you can see examples of that.  This alone solves a large part of the warehouse issue, although its a little more tricky in CHS where this stuff is encoded.
>
> I think off the shelf business intelligence products are better at solving the report creation problem.  Possibly with just using webservices to push pdf's or excel docs back into Sakai, that avoids some of the authz complications.  But more rich integrations using basic lti or linktool or whatever are conceivable as well.   It shouldn't take weeks to build reports or expose data in Sakai and in my experience even with a lot of knowledge about the system, that is what reports and the warehouse turned out to be in practice.
>
>
> On Mar 10, 2010, at 3:05 AM, Berg, Alan wrote:
>
>> Hi Dev and production list,
>>
>> I have  recommended to the Product Council and the Maintenance team that the reports and warehouse tools are demoted from Sakai 2.7 as Enterprise core tools and placed back into contrib. Please feel free to comment and make your views known.  The  original email sent is given below
>>
>>
>> > > Hi Product council and Maintenance team members,
>> > >
>> > > the MT are meeting Wednesday to discuss depreciation recommendations for the
>> > > PC as part of the Sakai 2.7 life cycle. This is good as the MT are the
>> > > experts who deal with operational reality on a daily basis.
>> > >
>> > > As Interim QA director I have the professional role to signal issues
>> > > clearly. The Reports tool does not work in the QA tag Sakai 2.7 beta 4 and
>> > > that the warehouse tool also appears to be unsupported. After discussion
>> > > with Noah and Beth I believe that the tools do not have sufficient community
>> > > support and are not being used widely enough to justify continued QA
>> > > effort. I therefore strongly recommend moving the tools from Enterprise core
>> > > status back into contrib before the beta 5 tag is released and thus giving
>> > > QA enough time to verify that removal was safe before 2.7 is tagged.
>>
>>
>>
>> Alan
>>
>> Alan Berg
>> Interim QA Director - The Sakai Foundation
>>
>> Senior Developer / Quality Assurance
>> Group Education and Research Services
>> Central Computer Services
>> University of Amsterdam
>>
>> http://home.uva.nl/a.m.berg
>>
>> _______________________________________________
>> management mailing list
>> [hidden email]
>> http://collab.sakaiproject.org/mailman/listinfo/management
>>
>> TO UNSUBSCRIBE: send email to [hidden email] with a subject of "unsubscribe"
>
> _______________________________________________
> portfolio mailing list
> [hidden email]
> http://collab.sakaiproject.org/mailman/listinfo/portfolio
>
> TO UNSUBSCRIBE: send email to [hidden email] with a subject of "unsubscribe"
>
_______________________________________________
management mailing list
[hidden email]
http://collab.sakaiproject.org/mailman/listinfo/management

TO UNSUBSCRIBE: send email to [hidden email] with a subject of "unsubscribe"