What value does lecture capture provide? Show me the numbers.

Lecture capture systems aren’t cheap. Sure you can do it cheaply, but not cheaply and well. If you want to do it right, you’re going to spend a fair amount of money. When you ask for hundreds of thousands of dollars, people are gonna ask questions.

  • What value is added by having such a system?
  • Are educational outcomes improved?
  • Do grades and test scores go up?
  • Can faculty be more innovative?
  • Will students attend class less?
  • Will students not participate in class discussions?
  • Will being recorded change the way I teach?

These are all great questions. Important ones. The question about whether we should provide this service at all is a great one. I’m sure there are other changes, more pedagogical in nature, that could yield better results.

  • Flipping the classroom.
  • More group work.
  • Shorter lectures.

I can’t control any of that. I’m just the IT guy. I’d like to make sure that lecture capture is effective and an appropriate tool for the job of improving student outcomes.

What amazes me is the lack of peer reviewed research data available on these topics. It seems to me that if I made a lecture capture product that purported to improve the student experience and enhanced learning, I’d have a lot of data and research to back that up. Instead, the best I have is some research done by peer institutions that they’ve shared with us. The lack of research is almost conspicuous in its absence.

Here’s where I flounder. Researchy stuff.

I've been compared to Steve Buscemi before.
I’ve been compared to Steve Buscemi before.

It seems that there should be some research that state: class A didn’t have the system and class B did. Class B scored X% higher on tests and received grades that were Y% better. Where is that data?

If one were so inclined, one could take a course where lecture capture was available and try to correlate grades with usage of the system. Unfortunately that’s a pretty poor test because correlation != causation. People that use the system more are more likely to be good students and be studying more, therefore such data may not be useful.

What I truly need is one professor teaching multiple sections of the same class in the same semester. Then I need to use lecture capture in 1/2 of the classes. I also need some sort of pre-assessment to determine how each class progresses. It’s far from perfect but it would be pretty informative.

Now here’s the deal. I’m not that smart. I’m not the first one to think of this. But has anyone actually ever done it?

If so, why doesn’t every manufacturer lead with it?

If not, why not?

I’m not somebody with a background in research and creating this sort of stuff. I’m just an AV/IT nerd who’s pretty curious. Help me out here?

The pain of procurement.

Maybe this is a taboo topic but I think it need to be discussed.

Our contract approval and procurement process could use some improvement.

The workflow for buying a product or service that requires a contract or that includes any sort of agreement goes something like this (at least from my perspective and experience):

  • Get contract from vendor.
  • Have OIT contract person review.
  • He asks for some basic changes which are sent to the vendor.
  • Changes are negotiated and either made or not made.
  • Send back to OIT contract person.
  • He forwards to general counsel for review. (All contracts, even for a product evaluation and free software, need to be run through the office of general counsel.)
  • They sometimes suggest further changes.
  • Negotiate the changes. (There are often things they won’t change.)
  • Tell general counsel that we have to have this and the vendor will not agree to the changes.
  • Get contract approved by general counsel. (Most of the time.)
  • Get someone at the VP level to sign. (Usually the CIO.)
  • Submit signed contract to vendor to countersign.
  • Get signed contract back.
  • Get vendor added to the purchasing system.
  • Get billing and tax information.
  • Determine which account is going to get charged.
  • Submit a PO using the signed contract.
  • PO gets approved by 2-4 people depending on the value.
  • PO is sent to the vendor.
  • Check with vendor to ensure that they received the PO.
  • Finally receive product.

If you’re very lucky and have recently sacrificed a bucket of chicken to Jobu, this is a 4-week process before the PO is sent.

jobu

Not included in the steps above are the inevitable delays in responding to email, voicemails that aren’t returned, vacations, sick days, etc. This happens on both sides of the coin. ND and the vendor. Often I’m the hold up because I forgot some crucial step and now the process is on hold.

Banging your head on your desk, crying and drinking are also not on that list.

If you need new software installed on a large number of classroom computers you really need to allow 3-4 months if you want to be 100% sure it will get done and work well.

Once the software is approved, we have to create a deployment strategy. We have disk protection on the machines so changes users make get erased at reboot. This can cause some issues with software deployment. It’s not just “blast out the msi and call it a day”. Sometimes you have to deploy the software then deploy fixes and/or additional files. It’s tedious in a multi-user environment. I’ve done the job and it’s not easy. much of the time these requests come in at the last minute as the image is being finalized so the software has to be pushed later, not as part of deploying the image. On more than one occasion I went through weeks of effort and then found out that the faculty member never used the software. Sigh.

Right now I’m asking for a freeware open source product called curl to be approved for use on a few machines but potentially all machines in the classrooms. The question that gets asked every time you need something approved is, “When do you need it?” “If you need it sooner than 20 business days, please explain why.”

20 days to get a freeware open source product approved just seems like too long. It seems like there should be a faster track for free and open source stuff than for complex contract negotiations.

Also, every time a new version of a piece of software comes out with a new or different license, it has to be re-examined and possibly re-approved.

At various times I’ve heard differing opinions about whether or not anything on the Apple app store needs to be approved before purchase and installation on a university owned device. Same with the chrome web store. Extensions, apps, etc. all need to be approved if it’s going on a device that Notre Dame purchased.

This all seems like a completely unscalable and unsustainable system. It seems as if we’re actually relying on people to not get everything approved so we don’t completely bog the pipeline down and break the system.

I understand that we have these processes in place to protect ourselves and to mitigate the risk. I’m just not sure we’re accurately assessing the risk. We seem to treat the risk of a lawsuit as the same across all products and it’s not.

Compare the risk associated with a faculty member installing Explain Everything on his iPad, to that of purchasing and deploying a new data analysis software package on classroom PCs. The risk profiles are completely different but the process might be the same depending upon who you believe. At the same time, we can’t just suggest to everyone in our organization that they install some Chrome plugin that’s free for personal use but not for business use. Untangling that can be very difficult. Sometimes it’s even hard to find the license agreements.

Here are a few problem this approval process creates:

  • People ignore it or are unaware of it: Sometimes willfully unaware. Faculty buy software all the time using their discretionary funds or just using their own money. Staff and faculty alike install extensions, apps and freeware on their computers all the time. None of that ever goes through this process and we’re now exposed as a result.
  • The time lost is huge: I’m now at a standstill on my project until that software is approved. Of course we could operate on the theory that “what is the likelihood that the software will be approved?” and proceed anyway.
  • People don’t feel free to experiment: I know that there have been instances where people just don’t want to go to the hassle and don’t bother.

I’m no procurement expert but I have to believe that the software and contract approval process could be sped up and improved upon. The idea that it can’t is almost too much to bear.

I_Want_to_Believe

I guess there are worse ways to spend $40 million. Not many, but they do exist.

Miami-Dade To Give Teachers 18 Months of Interactive Whiteboard Training — THE Journal.

Miami Dade is deploying over 10,000 interactive whiteboards by the beginning of the 2015 school year. Admittedly I haven’t priced smartboards in a few years. I sent emails to a number of school officials and haven’t heard back on the details of this contract. In doing a couple of quick checks, $4000 is a reasonable estimate for purchase and installation of one of these smartboards. At 10,000 units that’s $40 million dollars. Plus the 18 months of training. Plus bulbs for the projectors. Plus support.

It boggles the mind.

This is approximately $80 million
Approximately $80 million and Miami Dade doesn’t sell meth

That’s a lot of money spent on technology that hasn’t been shown to have any long term impact on learning outcomes.

I wonder how many of them will still be used as anything other than a whiteboard and projector in 2 years. Every one I’ve seen has been ripped out or gets used as a simple white board.

Some sales guy must have had a really good quarter though.

See also: YOU CAN HAVE MY CHALK WHEN YOU PRY IT FROM MY COLD DEAD HANDS!

 

Great instructors don’t make great performers

Making instructors comfortable with on-camera roles | University Business Magazine.

This article is a really good reminder and primer on some of the challenges related to creation of flipped content or distance learning. You can be the best instructor in the world and if you’re not comfortable on camera, people will have a hard time watching you and therefore will struggle to learn. Making faculty comfortable, reminding them of the goals, getting them to relax. These are really key steps in producing quality content.

The goal is for the digital content to be the same as it would be in person. Natural. Not wooden and stiff. Real emotion in your voice. A little excitement. None of that is easy or natural when you’re looking into a camera. It’s kind of like writing. I’m much funnier in person (thinner too!) but it’s hard to translate that into the blog because it’s all one-way dialog.

This might be a little too dramatic
This might be a little too dramatic

Recently I saw a pretty neat method of creating some flipped content. As opposed to just talking to the camera, they used a small studio space and had 12 students sitting in the room. The professor was describing a topic but was engaged with the students. There was Q&A and some real dialog. It didn’t feel canned or forced at all. The engagement with viewer of the video was enhanced because there was genuine engagement with the students in the room. Great actors and performers can fake this but it’s hard to do. This made it much easier.

We’re actually in the very early steps of doing some testing for a few different types of recording studios. (There a plan to have some studios in one of the new buildings being built around the football stadium but that’s 3 years away. It may as well be vaporware at this point.)

I see a need for three types of content creation:

  1. Traditional videos. With or without slides that appear over the speaker’s shoulder or in front of a green screen.
  2. Podcasting or computer and audio. Think Khan academy or YouTube how-to videos.
  3. 2U type videos where the instructor uses fluorescent markers to write on a piece of edge-lit glass that sits between he and the camera. (The image gets flipped either in post or via a mirror.)

We’re actually working on a small pilot of the last one. Should be up and running by mid-march. Once we can show it off, we hope to be able to secure funding to build it full scale and find space to house it.

I’ll make sure to take some pictures and document the build process.

Stay tuned!