Jim Kenney's ARP blog

Analysis of Cycle 3

2005-05-20T20:10:00.000-07:00

I have spent the past week collecting and analyzing the data for my cycle 3 synchronous class. I am really pleased to have so much data available. With pre and posttest results, post-course survey data, and results from personal interviews, I feel pretty confident in my assessment.

So what is my assessment? I have little doubt that synchronous instruction can be just as effective as traditional classroom instruction. The feedback shows that students appreciate both the convenience and the fun of being online. And although the test scores were a little lower this time, they are still comparable to classroom results.

The technology worked extremely well given the wide range of experience. In fact, of the ten students I interviewed, none had ever taken an online synchronous course. Therefore, it is amazing we only had a few technical issues. I am still surprised with how fast students adapt to the technology. The technology itself is almost a non-issue and of little concern.

So from a positive perspective, the approach works. It allows for an efficient way to spread knowledge around the country (and potentially around the world). Our participation from 5 locations proves the tremendous reach and effectiveness of this method of instruction. Here are some other benefits:

As already mentioned, the students like the approach and technology.
The technology is at a point that it is dependable and easy to use.
A flexible schedule with multiple and shorter sessions works very well for our audience.
Once the students learn the technology, they often feel more comfortable and relaxed then being in a classroom. Because of this comfort level there may be even more participation, as long the course is designed properly.
Synchronous instruction is very efficient in terms of cost, timeliness and access to expertise.
A learner-centered approach is workable if properly planned. We used the last two sessions to have students problem solve troubleshooting scenarios and then present their approach online to the class. It worked great! In hind sight, I wish I had desinged even more learner-directed instruction for this topic, but it was a successful start.

Although, there were many positive aspects of the last class, we also had some real challenges. Personally, I feel there is still much to learn before this type of instruction begins to reach its full potential. This last session offered many lessons to me as a designer and researcher. Here a sample of what I learned:

The instructor needs to be prepared and very familiar with the topic. This is even more critical with highly technical topics that our instructed online. Even with a learner-centered approach, the instructor need to be ready to step in and provide support.
The material needs to be tailored to the audience. In our case, we had several participants who did not have the appropriate experience or technical knowledge required as a prerequisite.
The learner's environment for taking the class must be quiet and free from distractions.
With online learning, students need to be more disciplined in arranging their schedules to allow for sufficient self-study and class time. Supervisors should also be responsible in helping their staff set-aside time for training.

Despite these and other "bumps" in the road, I see a tremendous potential for synchronous instruction for a variety of topics.

Cycle 3 Course Analysis: Day 5

2005-05-12T14:31:00.000-07:00

This was the final day of our course. To be honest, I am very relieved to have it over.

The fact the course was completed over 5 days really seemed exhausting and demanding from an administrative standpoint. This is something important to remember given this type of schedule.

I question whether one of our training groups has the ability to support this type of training. Since this was my project, I devoted a tremendous amount of time preparing and supporting the instructor. The fact that the instructor was still uncomfortable with the technology added even more demands on myself. There is no doubt that synchronous distance learning requires tremendous coordination and communication.

We finished today's class by having students present more troubleshooting scenarios. Again, this overall approach of handing the instruction over to the students worked very well in a synchronous environment. We ended the day by answering questions and completing the post-tests.

We had twelve students complete the course (of 17 enrolled). One of the twelve was a previous instructor of the course. Although he decided to not take the exams, he did participate in all the sessions. Most of those who did not complete the instruction had schedule conflicts, so this is not an area of great concern. It did concern me was the lack of consistent attendance of some students. Although each person had major work related conflicts that impacted attendence, the lack of participation must have impacted their learning.

I am still tabulating the scores, so more analysis will need to be completed later. I also have individual interviews set-up next week to gather feedback. It does appear from the initial scores I've seen, that this class will scored lower then the previous session. I will need to study this further and consider the feedback I receive from the class. I also sent-out a post-course survey that should also shed some light on the results. After assessing all of this data, I should have a clearer picture of the results.

What worked today:

Again, the Subject Matter Expert really help answer questions and clarified concepts.
We ended in plenty of time, and the students were able to complete the post-test.
As always, the technology seemed to work great.

Improvements needed:

This instructor needs to better understand the content and technology.
We should have added a final exercise that would have allowed the students to review content. Having them deliver this exercise themselves would have worked very well.

Final thoughts:

I thought our effort to add learner-centered activities in the end worked well. The next step is to add this approach earlier in the course. The key to synchronous instruction is involvement. I didn't feel hit a high level of involvement during this cycle. One major factor was the instructor. He simply did not have the expertise to facilitate this course. I thought the instructor for our second cycle was more effective because he new the subject matter, and as a result could better facilitate discussion with the class.

Cycle 3 Course Analysis: Day 4

2005-05-10T18:31:00.000-07:00

Today we had a portion of the class instructor led facilitation, and the remaining time devoted to student presentations.

My hope was that by turning the delivery over completely to the students, we would move toward a more student-centered approach, and move toward less dependence on the instructor. What was interesting was the contrast between both approaches. The students, especially for those delivering, performed very well and seemed to benefit from the whole experience.

In contrast, the instructor led piece seemed to drag along, and involvement was minimal. Even though the instructor attempted to ask more questions that would improve dialog, it was difficult to get the amount of discussion desired. I believe part of the problem was the instructors motivation today. He didn't feel well for one. He also had several stressful meetings planned for later in the day. During the student's presentations at the end, he even excused himself to attend another meeting.

On the positive side, the student-led presentations went very well. The students led the class through troubleshooting scenarios and explained the choices they made. In the end, my subject expert offered some very effective feedback. Tomorrow, we will spend more time having the students present. My hope is that we will again have some positive participation.

What worked today:

The student led presentations.
The support of the subject matter expert. He provided just the right amount of support and feedback.
The timing and pace of the course seems appropriate today.

What should be improved:

The instructor must be better prepared and appear confident throughout the presentation
When minor changes are made to the course, it is not beneficial for the instructor to share the changes with the students. For example, he noted several times that we decided to leave out certain polling questions to save time and because they were not necessary. This probably distracted the students and hurt the perception of the course.

Cycle 3 Course Analysis: Day 3

2005-05-06T15:06:00.000-07:00

Having our Subject Expert (SE) return to today's session made a real impact. It also reinforced my concern that the instructor may not have enough expertise to effectively deliver the content independently. This weakness seemed to hurt the interaction. The SE, on the other hand, seem to invite more participation and questions. When he was called upon, the interest from the students quickly increased.

So what is the lesson here? The online instructor should not only be an excellent facilitator, but he or she should have a sufficient level of expertise about the topic. This seems to be important even if a SE is available to assist.

We had 7 students missing today (of 16 enrolled). This was due to several work emergencies the pulled away many of the students. This lack of attendance might have also hurt the participation and momentum of the session.

Work worked well today:

Our SE was able support much of the discussion and questions.
We introduced next week's activity that will allow the students to lead work through a scenario and present to the group. Everyone seemed positive about the exercise.

Improvements needed:

The instructor was moving through the content too quickly at times. This was the opposite of what happened with our previous instructor who too slow.
As already mentioned, the instructor needs more expertise with the subject matter.

A final thought I had relates the different approaches and personalities of instructors. I chose the instructor for this cycle because was more energetic and engaging in his approach. However, he does lack subject knowledge and is less comfortable with the technology. My instructor from the previous cycle had a much stronger subject knowledge and seemed more relaxed instructing online. Unfortunately at times he seemed too passive when delivering. In the end, I appears that online instruction needs a little of both types.

Cycle 3 Course Analysis: Day 2

2005-05-05T18:44:00.000-07:00

It is amazing the similarities between this course and my previous one (cycle 1). In both cases the primary concern is with the instruction. Both instructors were awkward with using the technology. With my current instructor, the uneasiness with the technology seems even greater. The fact that I am helping him more in managing the technology may even be hurting his learning curve. Is he becoming too dependent on me? Should I have let him struggle with the tools more from the beginning? I'm not sure.

My hope was that by supporting him with the technology, he would be free to focus on facilitation. What's interesting is that he does not seem as prepared in delivering content. The previous instructor seemed more comfortable with the subject matter. My current instructor seems to struggle with every question.

One surprise today was the absence of our subject expert (SE). He told me that he would be late, however, because of some urgent work issues, he had to miss the entire session. I quickly realized how dependent the instructor was on the SE. My instructor simply did not have answers for challenging questions. Does this mean I picked the wrong instructor? Is it too difficult depending on two key facilitators? I believe the answer is somewhere in the middle. I need an instructor who is prepared enough to handle content alone if needed. However, there is also a benefit in having a SE ready to support an experienced instructor.

I had another concern that was similar to the last course. We again have a wide range of abilities represented in the students. The fact that the instructor struggled with the content, and appeared to lacked confidence, probably isolated both beginners and experienced students. I decided to try something to remedy this situation. I asked my SE if he could come to the next class an half-hour earlier, and be available to answer questions. I also asked him to stay a half-hour after class to answer additional questions. I sent out an email today informing the students of this added service. My hope is that some will respond and use my SE's expertise.

We had 13 attend today (out of 17). Since several student were unavailable because of shift challenges, and one was on sick-leave due to back problems, I felt the turnout was acceptable.

Tomorrow will be interesting. I plan to introduce an activity that will allow the students to present their own chosen troubleshooting scenario. This we be a major step in making this course more learner-centered. The students will choose their own scenario and present to the group a strategy next week. I am very interested to see how this will work. It should take the focus and pressure off the instructor and allow the learner to take more control of the learning. It should be interesting.

Here is some quick thoughts about this session.

What worked:

We had more interaction/questions today from some of the students.
Despite having more questions, we still covered most of the content needed.
We started on time and had all participants log-0n successfully (although one student had to use a loaner computer because his computer had problems downloading a needed plug-in).

Challenges:

The instructor was not adequately prepared to deliver the content.
The Subject Expert was unable to attend and support the instructor.
The instructor was still learning the technology.
There seems to be a wide variety of skill levels represented.

Cycle 3 Course Analysis: Day 1

2005-05-03T18:05:00.000-07:00

For some reason, I was more nervous about the start of this cycle.

My previous cycle involved implementing our first synchronous course. Maybe it was because we considered the first course a pilot that I felt less stress. Calling our first effort a pilot took the pressure off trying to make the course mistake-proof and lowered my end-result expectations.

Of course, the anxiety this time may simple be caused by the start of a new OMET term and a demanding work schedule. One thing is for sure, I am still pleased to get started with my final cycle.

We had 14 students in attendance today. Considering we had 16 enrolled, I felt pleased with the exceptional turnout (similar results to our first cycle). We had one person unable to access the web-conferencing tool. Although he was able to listen to the audio portion, it was frustrating that he was unable to get online. I plan to meet with him early in the morning to work on the problem. Hopefully with the help of the web-conferencing technical support we will be able to solve the issue.

What went well:

As I already mention, we had an excellent turnout.
Nearly all of the participants logged-on early. This helped the course get started on-time.
The move to an hour-and-a half session worked great (cycle 1 was one hour). This session feel less rushed and we finished on-time.
We had more questions and discussion during this session compared to cycle 1.
We had all 5 sites represented (compared to 3 for cycle 1). It was really exciting to see such a diversity of representation.
Since both myself and the subject expert were in the same room as the instructor, we were able to provide more support and keep the pace moving. This helped the course go much smoother.
Except for the one student with technical issues, all other students easily used the technology.

Improvements needed:

This was the first online course for my instructor, so he was still awkward with the technology and the need to multi-task. We simply needed more preparation time.
I need have a better plan in place to help those having technical issues. I felt unprepared for the one person who was unable to get online.

Overall, did I feel this was a successful start? Yes, considering we had an understandably nervous instructor. And as I stated earlier I was also nervous. It is difficult to move from a pilot to a regular session, with all the added expectations. The truth is, I feel this whole approach will continue to be a learning experience for years to come. The pilot may never end!

Student Prep. & Cycle Differences

2005-04-25T21:24:00.000-07:00

As with my previous cycle, I have been investing time preparing students for the upcoming online session. There are similarities to the last class (cycle 1), but there are also differences.

For one, the class size is currently at 11 students. That is quite a reduction from the previous class of 16. Hopefully this adjustment will encourage more participation. I have also tried to better prepare students for the challenges of online learning. This has included recommendations for creating a more quiet environment and one free of distractions.

Finally, I have worked with a new instructor to prepare him for more dynamic online sessions. I will continue over the next few days to improve the course content, including adding an approach that is more learner centered.

There is one final difference worth noting. I will have 6 students from our Puerto Rico site. I'm not sure how having this new group will impact the results. Will their challenges in understanding English become a liability? Will their participation be more or less then that of other students? I don't know. It will be interesting to evaluate the effectiveness of this type of delivery for the Puerto Rican culture.

Preparing for Cycle 3

2005-04-08T10:35:00.000-07:00

I can't believe how fast time is flying in this program. I am beginning to prepare for my 3rd cycle, and I anticipate a whole new set of challenges as I prepare for my 2nd online synchronous class.

First, I will have a new instructor. My new instructor should be ready to instruct, but in some ways I feel I am starting from scratch. As I mentioned in my cycle 2 report, there is a certain lack of control in this type of project. In the end, it is up to the instructor to enable the students to learn. I can help prepare the instructor, but it is ultimately up to him to guide the learners.

Another concern is the actual need for the course topic. Those in our organization, especially those who would be attend this class; have been impacted by other organizational challenges. These other problems, related to a new system being implemented, could result in few seeing my class as a priority. So what would I do if the attendance is too low? As always, I need to learn to cross that bridge when or if it comes. That is the risk of making an intervention, it may be poorly timed.

On a positive note, we have a strong instructor and SME available to deliver the instruction. The training lab and equipment are also available. All I need are the students and were set. I sent out the course announcement today. I now have to wait to see what need there is in the organization.

Course Analysis: Day 6

2005-03-05T19:12:00.000-08:00

This was the last day of class, and I am quite relieved. One challenge of our online curriculum, is that it lasted for six days, and over two weeks (held on Tuesdays, Wednesday and Thursdays). Even though each session only lasted an hour, there still seemed like a significant time commitment. Of course, if you include the preparation and administrative time required, the time commitment was actually rather demanding.

For day six, we spent the first half-hour reviewing the remaining troubleshooting scenarios. The instructor worked harder this time to ask students to explain their answers. I thought this stratedy helped, but more was still needed.

For the second half of the class we allow the students to take the post-test and complete the feedback form. The instructor and I stayed online for anyone needing help. The post-test worked fine, but students were unable to access the feedback form. I didn't discover the problem until early the next day (the link was distorted when I used certain email addresses). This was unfortunate since I doubt most students will complete the form days after the class. I will have to wait and see who responds. Since I anticipate a low response, I have decided to schedule short phone interview to gather feedback. It will be interesting to see if this strategy works, or if students will be resistant sharing feedback.

I did learned a valuable lesson from my feedback link problem. Test everything! Anything involving technology should be check to ensure it works exactly as planned.

The good news is that the post-test results were very positive. In fact, the post-test scores on the average were just slightly better then the averages of all previous face-to-face classroom sessions. The results seem to support most of the research I read about in my literature review (no significant difference between online and traditional instruction). Considering that we had 16 students representing 3 sites, it would seem to indicate that this was a very effective and efficient method of instruction. Will the students prefer this type of instruction compared to the classroom alternative? I will not know that for sure until I get more feedback.

In the end, I still feel the instruction needs to be improved. Much of this should come from an improved curriculum. A more constructivist approach, with small individual and small group activities, should be explored. In addition, the instructor's facilitation skills need to be adapted to a synchronous approach.

Did this synchronous instruction help encourage collaboration across sites. No, not to any great degree. Again, a change in the curriculum could encourage more cross-site communication.

In summary, here are some major areas that need to change:

Facilitation skills are needed that are more learner-centered
Optional areas are needed for students when noisy conditions inhibit learning
A curriculum needs to be modified to encourage more individual and group participation

There are more, but for now these are the big ones to consider.

Course Analysis, Day 5 (APR Cycle 2)

2005-03-02T15:21:00.000-08:00

I was pleased that the two students missing from yesterday's session had returned (16 of 18 in attendance today). I also thought that the pace of the class was faster and the participation more energetic. These were all very positive changes.

This was the first day we introduced troubleshooting scenarios to the class. We decided to make moving through the scenarios very interactive. Therefore, we called on each student to provide answers to scenario questions (each scenario asked students to choose the best course of action for a problem situation). Since there is no right or wrong answer to each question (only a better course of action among choices), we felt the activity would be non-threatening. Overall, we were pleased with the results. It did help get student involved in the discussion. However, our concern is whether the students are really thinking about their answers and learning from the scenarios. We decided for tomorrow scenarios, we would ask more "why" questions to help student explain their answers. For example, "why did you choice action A."

We were also surprised by the speed we moved through the sceneries. We moved through all 5 scenarios in less then a hour. Originally, we thought that we would only make it through 2 or 3 scenarios. Why was this? It may be the discussion is still lacking. Even a few more questions would have impacted the time needed. If we add more "why" questions, I'm sure we would slow down our pace through the scenarios.

Course Analysis, Day 4 (APR Cycle 2)

2005-03-01T14:28:00.000-08:00

Prior to today's session, I sent out a short survey to all students with the intent on gathering some mid-course feedback. There were several concerns mentioned in the survey that caught my attention.

First, the pace of the instruction may me too slow. I think this is a valid concern. This instructor seems to lack the energy and flow needed to hold the attention of the students, especially for our online format. It appears that this type of instruction may require a more assertive rather then passive instructional approach (affirmed also by my research). I definitely feel that the pace is impacting participation and interest. Could the curriculum be at fault also? Yes, the curriculum does need to be more learner-focused. However, I still feel that the instruction is a major issue.

I did review the survey feedback with the instructor prior to today's session. We both agreed that the pace should be increased. Did he make the adjustment? I thought the pace was better, but still lacking. My gut feeling is this type of delivery is not a good fit for my instructor. Obviously, I will need to be patient and look for more feedback at the end of the class.

Another concern are the distractions and noise levels for the students. One student already complained that there was too much noise at his desk. Since many of our students are technicians, system owners, and operators, they may continue to have issues with noisy work areas. For cycle 3, I may want to look at alternative areas for students to take the instruction. This may mean having students taking class from conference rooms (plugged into the network via laptops). Although this may be less convenient, it may also provide a environment more suited for learning technical content. This change would also allow for students to take the instruction together in small groups, and allow for greater collaboration and team activities to augment the online instruction.

My final concern was the attendance today. There were 4 students who missed the session. We will need to discuss what action to take if attendence also slips tomorrow.

Course Analysis, Day 3 (APR Cycle 2)

2005-02-24T11:43:00.000-08:00

We had a full class today which was amazing. The entire 18 students were all present and logged-in. Again, the continued high attendance was very encouraging.

What worried me was the lack of participation. For some reason, we had less participation (in the form of asking and answering questions), then on either of the previous two sessions.

Here are some possible reasons for this decline:

The content was very complex with many process charts. The students may have needed time to process all of the concepts.
There may be some intimidation with the difficult subject matter. Is it possible that student may fear asking a dumb question or providing a wrong answer?
Is the pace either too fast or too slow? If it is too slow, then we may be losing the attention or interest of the students. If too fast, then students may be lost or confused.
Maybe there is still a intimidation factor of the technology. The students may not feel comfortable using technology (raising their hand, chatting etc.) as a means to participate.
Finally, maybe the instructor is not using the right technique(s) to get involvement.

After discussing participation with the instructor and subject expert, we feel the second point above is a likely possibility. For one, we seem to have a few who answer most of the questions (they also seem to be the most experienced. Also, it appeared even those who answered questions seemed to hesitate at times.

It is also possible that the pace is too slow at times. The instructor's occational long pauses may be a factor (or at least a distraction).

So here are some actions we plan to take:

The instructor will emphasize the importance of participating and feeling comfortable providing wrong answers.
He will also (in a non-threatening way) try to recognize and question individuals.
I will send out an email asking for feedback for the course so far, and to solicit questions about the content.

I will send suggestions to the instructor prior to the next session.

Course Analysis: Day 2

2005-02-23T19:18:00.000-08:00

I was curious at the start of today if we would loose any students. After all, this whole approach to instruction was new to probably every student. Thankfully, our attendance was great! In fact, we went from 15 students (for day 1) to 17 students. That is one less then the initial list of 18 whole enrolled! This is almost unheard of in any of our classes (especially those requiring a longer time commitment). It will be interesting to see if the attendance remains constant. Stay tuned!

I was also concerned about today's class because the course content was quickly becoming more technical and complex. I thought everything went well except we ran out of time (one of my early concerns). My instructor and subject expert both felt fine about the pace and lack of progress in staying on schedule. They also felt the content was difficult enough that a slower pace was warranted. I agreed. There was also increased participation, a worthy tradeoff for more content. Will we be able to make-up ground in covering content? We don't know. However, in taking a lesson from my current OMET studies - maybe quality rather then quantity really applies here.

Here are some more details about today's session:

What went well?

As I already note, the attendance was very positive.
Great participation from many of the students. They asked a far greater number of questions via the chat feature and by raising their hand then the previous session. We hope for even greater participation tomorrow.
My subject matter expert provided some great input, and answered both questions via chat and vocally. He really freed-up the instructor to concentrate on instruction.
I was very pleased (again) with web-ex. All students (including the student who couldn't log on yesterday) were able to log on and use the different functions. I even heard a student cheer in the end that he had successfully used NetMeeting (who cares that he named the wrong tool!).
My instructor seemed more confident and handled all content very well.

Improvements?

The instructor still has some difficulty managing the pace of the delivery. There were a few times that the content could have been delivered little faster. Although, as I noted above, a overall slower pace was needed for most of content. Learning when to vary the pace of delivery will probably come with practice.
Overall, more time may be needed to cover the course content. At this point, we will need to pilot the remaining sessions to determine what timing changes are needed. Who knows, the end result could be to cover less content. It is difficult to determine at this point.

Tomorrow's class will continue to have complex content and concepts. I will be looking closely at the course flows and if there is adequate time for delivery.

Course Analysis, Day 1 (APR Cycle 2)

2005-02-22T13:25:00.000-08:00

We completed the first session (five to go!) of our synchronous course this morning. Overall, I was pleased with our first session.

Of the 18 students who had signed-up, 15 were in attendance representing plants in Ca. and Wash. (a very high attendance rate for our organization). In addition, one of the absent students (from Rhode Island) asked the instructor for a make-up session (which he was able to provide after our session). Therefore, we should continue tomorrow with 16 students. Besides the students, instructor and myself, we also had our subject matter expert and back-up instructor participate.

Of those students who attended, all but one were able to log-in to WebEx. The one student who was unable to log-in was still able to listen in via phone conference. I agreed to help him to log-in prior to session two. Hopefully I will be able to learn what happened and how to prevent the problem in the future. I did learn difficult it is to help someone after the training starts. I'm not sure yet what is the best method of helping someone when they phones are tied-up. I will need to address this later and develop usable strategy.

The following is a summary of what went well and what needed improvement:

What worked:

The attendance. As I already noted, it was very pleasing to have high attendance especially after a company holiday (President's Day).
All students were able to get logged-in to Web-Ex except one. From a technical standpoint, we had very few problems. The main issue was on my end. I had only one network connection working in my conference room resulting in the need to share the computer with my subject matter expert. This was merely an inconvenience and did not impact the course.
The instructor successfully made it throughout the session. My main objective for the first class was simply to make it through the session with no major problems. I was very proud of my instructor! At times there were long pauses and the overall flow was a little choppy, but I was still very impressed with my instructor's first synchronous performance.
The timing for day 1 was perfect. We actually ended about 5 minutes early, which I thought was fine for the first day. It will be interesting to observe the timing for session 2 since the content will be more complex.

Possible Improvements:

I had hoped for more participation. The instructor asked plenty of questions, but the responses were minimal. There may have been several reasons for this. For one, the new technology may have affected their comfort level. It may take more time for the class to be at ease with the new approach. Second, there may simply be too many students in the class, especially for such a technical topic. I should have a better feel about these concern after tomorrow's sessions.
Related to the above concern, I discussed with the instructor possible strategies to encourage more participation. This includes asking more questions that will require everyone to respond (using chat or a poll). I also suggesting calling on two or three participants from a site to answer a question (this would not single out a single person, but still encourage more input from certain individuals).
The instructor seemed take too long in uploading a poll. For some reason, I am unable (as a host) to perform this function on his behalf. I will need to look into this further. This may end up being a function that only the instructor (presenter) can perform. In that case, it may just require practice for the instructor to become faster.

I am looking forward to tomorrow's session. The content is more complex for day two, so I am interested in how the increased difficulty will affect the timing and participation. For now however, I am just relieved to have day one complete.

Enforcement & Bureaucracy

2005-02-20T17:09:00.000-08:00

I went into work on Saturday to do some some last minute checks on my course preparation. Is my training room ready? Are there any last minute emails related to the instruction? Are there some tweaks that still need to be made to course materials? Basically, I just wanted to my mind at ease.

Fortunately, I decided to make the trip to work. In answer to my first question, I was shocked to learn that the training room was not ready. My plan was to use this room as a sort of control room for the synchronous instruction. I would have several computers available as a way to help the instructor administer the class. The problem was that the conference phone wasn't working (rather critical for a course requiring phone conferencing). Several "urgent" work orders had been put in the previous week to get the phone fixed. So of course I was shocked when I found the phone still inoperable. Thankfully, I was able to locate another room that was in working order (a miracle for sure, thank you God). The challenge will be finding a location (or getting the crazy phone fixed!) for day two. I will have to live by my motto "one day at a time" for at least the first few sessions of this course.

I was also surprised to learn that one of the enrolled students had completed the required the pre-test at the end of day Friday. Since I had required everyone to finish the test on Thursday, I was a little angered at the irresponsible tardiness. So what to do? I decided to look the other way, and let the law-breaker into the class. Actually, I really think it was my whole OMET experience that changed my thinking in this area. After all, the purpose of this online is to meet the "learners needs." And to be honest, this was not a policy that was required by the organization. It was my own way of getting what was needed. Is there time to enforce the rules? Sure. However, there are also times that the student takes more responsibility and lives with the consequences.

So what lessons did I learn from these pre-course challenges. For one, I need to have a back-up plan for all of my classes, especially synchronous instruction. I should have had an alternate location already booked for my control room. I also need take more caution when working our IT bureaucracy (no more assuming that something is fixed). As for my delinquent test takers (and other troublemakers), I need to relax and provide a more flexible learning environment. This will be especially true when working with technology. The fact is, most of my students are extremely busy and need to see learning as friendly and stress-free as possible.

Last Minute Concerns

2005-02-17T21:22:00.000-08:00

I completed the final preparation today for next week's class. However, I wanted to record several concerns that I have discussed with those involved in this step of the project (my instructor, subject expert and instructional designer). It is amazing all the anxiety that seems to surface on the eve before the big rollout. I guess this is the price of having a project with many uncertainties.

Our first concern is the length of each session. The class is currently broken into 6 sessions, with each session lasting 1 hour. The reason for these brief sessions is to ensure that students stay alert, involved and attentive. However, is one hour enough time to cover content and allow for questions and dialog? At this point, none of us are sure just how much time will be required for questions and clarification. We will have to wait and see, and possibly make adjustments.

Another concern are the differences across the sites. Since we have students representing three locations, each with differences in equipment and procedures, will these variations inhibit what really needs to be learned. Will the instructor be forced to generalize the content too much? In this case, I feel confident that the benefits of having multiple sites included far outweighs the differences. The whole idea of cross-site communication is powerful to me, and will hopefully lead to future CoPs. Of course, this does relate to my initial concern - is there enough time within a 1 hour session to have strong communication? Maybe not.

My final concern is the fact that we are holding the class over 6 days. Will it be too difficult for students to attend every session, especially with demanding schedules? In our course introduction we mentioned that attendance at each session is required for credit? Do we try to enforce this, even if a valid excuse is given? For the integrity of the course, and for learning purposes, I think holding the line here is important. One thing is for sure, this could be a difficult situation to address if it happens. Again, I just won't know if this was the right decision (the 6 days) until the results are evaluated.

So what's next? I will need to evaluate whether these concerns are valid by carefully observing each session next week. These very well may lead to changes for my 3rd cycle. In either case, whether changes will be needed or not, this will definitely be a powerful learning experience.

Pre-Course Details

2005-02-11T20:23:00.000-08:00

This week I was completely immersed in course logistics. I had forgotten how much administration is involved in course preparation. The fact that my course is online seemed to further complicate matters.

I was able to send out the pre-course performance evaluation survey and pre-test. I also sent out a course introduction and initial instructions.

All this communication gave me several concerns. First, would all of these pre-course tasks overwhelm the students? Two of the participants already cancelled after beginning the pre-test. They each told me that the content was not what they expected and that it didn't apply to their jobs. This is probably true, I just hope the pre-test didn't scare them away.

Second, do the students really understand how this "online" training is going to work? After all, this type of synchronous instruction is probably new to all of the students. The red flag came when one of the students asked why he needed a headset for his phone when the course is online. I realized that to him online meant one of our asynchronous elearning classes. He obviously had little understanding of how a web-conferencing tool really works.

So what can I do to address these concerns? First, I need to send out another email clarifying the purpose of the class, and carefully explaining how synchronous instruction works. Will this further confuse students? It might, but I feel the added clarification is more important. As long as I keep the email positive in tone and short in length it should be effective.

What else is needed before the class? I need to review my plan and ensure all logistics are addressed. To be honest, I really don't enjoy this part. However, I know that much of the success of training, especially with online instruction, comes from thorough planning and preparation.

Cycle Two Preparation

2005-02-05T13:29:00.000-08:00

I began revising my existing curriculum to meet the needs of the online class. I should have the revision complete by the end of next week.

The challenge, as I see it, is to make the online sessions as interactive as possible. This will be difficult with a class size of over 20. Did I allow the class to get to large? Maybe, but the reality is that I don't know for sure what is an ideal minimum or maximum for this type of instruction. This will be an area of learning, and possibly lead to an adjustment for cycle 3.

One area that seems to be especially important is the student preparation needed for the first session. Having the students read and complete initial activities (i.e. pretest, knowledge checks etc. ) will help ensure a more active online session, and help focus the instruction onto deeper concepts. It will also allow for a more flexible online session and encourage more questions.

I feel okay about the timing. I should be able to complete the curriculum this next week, and allow for plenty of time before the class. However, I am aware that any major delay could rush the development and hinder the success of the class. It is important that I maintain my focus and energy as I move toward the big date.

Gaining Momentum

2005-01-20T19:38:00.000-08:00

I went through a period of doubt or anxiety over my project the last couple of weeks. I can't pinpoint exactly why, just that I had lost the energy needed to move forward. I felt my momentum slowing. Part of the problem might have been the first cycle. It went okay, but seemed to really test my ability to both learn a new technology (the web-conferencing tool), and then to use that technology to facilitate a class. I guess the fact that I personally was challenged, naturally made me wonder how other instructors will adapt to the new technology. As with anything new, especially with technology, there will no doubt be a learning curve for everyone. This need for patience and flexibility is definitely a major lesson I've learned in OMET.

Anyway, I decided to act on my anxiety this week. I remember a teacher once saying that "action conquers fear." I have never forgotten that quote ( I use it often). So acting on that wisdom, I decided to finalize the schedule for my cycle 2 synchronous class. Although it took some time to nail down a date that worked for everyone involved, It felt great to finally have the course on the calendar. I then sent out an email to all sites announcing the pilot and asking volunteers. It funny, but I really underestimated the amount of interest in this type of training. In less then a week, I have signed-up over 13 students. My concern is no longer over having enough students, but over the possible need to limit enrollment. It's really a problem I didn't expect. So it does feel good to feel the project pick-up speed. The excitement is back!

Cycle One Wrap-up

2005-01-13T19:45:00.000-08:00

My first cycle is behind me and I look forward to my next challenge, organizing the delivery of our first online synchronous class. The instructor preparation course went fine (my cycle 1), but this next step seems enormous. Just the logistics alone seem daunting when compared to my other responsibilities.

So what is my main concern? I think it has to do with the course schedule I have chosen. Instead of having the course delivered in a day and a half, I am having the course delivered in 6 sessions (one hour each). From an instructional standpoint, especially for online learning, it seems like the smart strategy. Especially with my concerns over using an online tool and the complexity of the topic. But it is so new compared to any other schedule or approach that we've used.

Despite these concerns, I feel it is important to stay the course and follow my plan. I must remind myself that there is always risk, especially with an endeavor that is so new. I also need to be realistic about the outcome of the first class. This is a pilot, and I should be prepare for a bumpy road. And the truth is, I learn the most from bumpy roads.

I am encouraged by the level of support I've received. Just yesterday I received a confirmation that an experienced Subject Matter expert is a available to support the class. This is good news. Now for the final push to prepare and organize the next course. As far as support, the foundation is set.

Cycle 1 Summary & Feedback

2004-12-20T20:38:00.000-08:00

I was very pleased with the excitement and support that came from this instructor prep session. I believe the class helped prepare the instructors for future class. But, equally important was the willingness from everyone to proceed with actual classes. At this point, I feel the motivation is just as importance as is the proficiency with the tool.

Here is a summary of the feedback I received at post-course interviews conducted with each student. This was an informal survey that allowed each student to elaborate on the online experience.

All participants had positive about WebEx and felt it should be used for future online sessions. Several expressed that the tool was even fun, and they enjoyed using the drawing and participation features.
The participants did appreciate that they had time to "play" and practice using many of the features. However, some wished we had more time to using the tool to use actual real course material. Since we ran out of time, we did have little opportunity to review future course content. One participant actually requested more practice time in the future before instructing the first session.
We failed to have enough breaks. Since the course was held over several hours, at least on break was needed. More breaks will be needed in the future to ensure students remain attentive.
Despite the one instructor asking for more practice (using actual material), everyone agreed that we should move forward with the first pilot. We did agree that sufficient preparation time would be need for this to be a success.

11/16/04 Update: ARP Plan Changes

2004-11-16T08:50:00.000-08:00

After meeting with my project team, I have decided to delay my online course pilot until January. This is due impart because of the busy December course schedule, and the workload of my instructor.

However, an equal concern is instructor preparation. A major theme that has developed from my literature review is the need for quality instruction. My instructor needs to be prepared to use the technology and have practiced the techniques that are unique to synchronous delivery.

The next few weeks will be extremely busy. I would like to complete a draft of my literature review and complete preparation for the practice session. I twill be important to maintain the momentum on the project.

Its probably a good time to reflect some early lessons learned from the initial steps of this project. A key lesson is to be cautious about being too aggressive in implementing an action. I should have anticipated the December resource limitations and the need to further prepare the instructors and the team on the new delivery strategy. Going forward, especially for future cycles, I will need to consider the entire resource "big picture" and make adjustment early in the planning process.

Pioneering Spirit

2004-11-01T21:48:00.000-08:00

I was challenged by some of the research I read today. It spoke of the need for the Online teacher to have a pioneering attitude - one who is resilient, flexible and able to persevere. It is a trait that often I see as my opposite.

When I consider someone like Howard Gardner, I see someone who has developed and defended his theory for years. He has devoted his life, so it seems, toward promoting the concept of Multiple Intelligences. I admire people like Howard Gardner, who are not afraid to spend their lives charting new territories, developing new theories and defending what they believe in. They are the pioneers of education.

I want to fully develop that pioneering potential in my own work. My attitude toward my ARP, and my OMET journey in general, should be one of adventure and expectation. My ARP team also needs to accept the challenge to join the expedition, to boldly go where no one at Amgen has gone before. It seems obvious that for change to take place, we may need to try some radical ideas, to take calculated risks. If we do take this challenge, then where would the path lead? Maybe towards a company that fosters a variety of communities of practice. Or, quite possibly a whole series of synchronous courses online. The possibilities are endless! It really makes me feel like a pioneer viewing a vast wilderness of opportunities.

October 31, 2004: ARP Plan & Team Update

2004-10-31T16:43:00.000-08:00

ARP Plan

After thinking this through for the past month, I have finally decided to add an asynchronous component to the project. It is through my literature review that I have learned the benefits of a using variety of delivery methods, and the unique advantages of asynchronous learning. My learning circle recommended also adding the asynchronous option last month.

So why the change? For one, asynchronous learning will support my course audience and subject matter. Since my topic is so technical, it is only expected that many questions will generated by the students. With an asynchronous tool, the students will have the ability to post questions in a timely manner, and receive responses from peers, subject matter experts and the instructors. My hope is that this forum will go a long way toward encouraging information sharing across sites, and move the students toward a functioning Community of Practice.

I still have a concern about introducing two many changes during the first cycle. Since this will be the first web-based pilot for our area, I want to ensure a successful implantation. I also learned that the threaded discussion tool I wanted to use will not be ready until early next year. Therefore, my thought is to add the asynchronous tool during cycle 2 or cycle 3.

So my plan draft has been completed with the new asynchronous addition. I look forward to moving on toward completing my literature review and implementing cycle 1. It should be a very busy November.

Team Update & Reflection

I met again with my course implementation team. At the meeting, I was able to outline the web-based training and my project needs. Thankfully, I received the support for both an instructor, subject matter experts and potential students for the initial December pilots. Hurray!

My primary concern getting the course ready for the pilot. Can I do it? Will I be able to maintain the interest and support for the project? It is interesting, but I see how much of my energy and enthusiasm is needed to maintain the momentum of this project. It will be vital for me to balance the demands of the project, my other classwork, the family demands etc. But there is no question about it, I will need to believe in the outcome of this effort.

It important is to stay focused on the need for this change, and to consider the potential benefits, both for Amgen and myself. As I reminded my team this week, this is real pioneering work for Amgen, and as a result, there is a certain level of risk. But it is exciting to realize that I will experience tremendous growth, no matter what the outcome.

October 22, 2004: Team Support/ID Support/Research

2004-10-22T22:00:00.000-07:00

Team Support

I asked for project support at yesterday's Mid-Level team meeting. This was the team that is responsible for developing and supporting the topic I plan to use for my synchronous course.

During the meeting, I was unable to locate the presentation slide that would have detailed major points of my action. As a result, I had to explain the project and web-based instruction without visuals. I thought I did an okay job of communicating my plan, however I still wondered if I was clear in the details. Fortunately, they all expressed interest in the project and wanted to participate. Everyone agreed to a future planning session.

ID Support

I met with my instructional designer today to discuss modifications that would be needed to convert the existing course into a web-based version. We began by changing the course schedule. Our plan is to break the course into five or six online sessions, instead of the current single day-and-a-half classroom session. Overall, I feel very good about the progress of the course.

Research Results & Concerns

So far, I have found very little research related to online learning in Pharma. However, I have located other research comparing online learning to classroom training, especially in academic settings. Overall, much of the research seems to indicate that online learning is just as effective as classroom instruction. Another trend is the need for quality instruction. I will need to ensure that my instructors are adequately prepared for online delivery.

9/16 Update: Gathering Support

2004-09-16T13:44:00.000-07:00

I continue to communicate my project to those in the department and other clients. Here is a summary of the interaction I've had:

Paul has again showed support for the project. He seems concerned about providing too much guidance or feedback at this time. I told him I am open to his guidance and critical feedback at anytime. He has already provided two great contacts at work, one who has used the Webex tool for instruction, and another, Calvin, who can provide IS support.
Met with Calvin and Angie to review the project needs. Both are willing to help with the initial pilot(cycle 1). Calvin emphasized that the right instructional approach was more important than the technology. I agreed. He seems to have a good sense of what needs to be done. Angie mentioned that that the documentation for this training will need to be agreed to. She will the documentation issue at another meeting and come back with a solution.
Calvin and Angie agreed to support a test run of the conferencing tool. I would like to have the course instructor and subject expert also participate in the test.
I Met with Evelyn from Puerto Rico. She is supportive of the project. She suggested that we pilot the technology within POQ Training first. Her idea was to use measurement and evaluation as a topic. I will discuss this with Chad. My preference at this point is to use the autoclave class as a pilot. It seems like a better fits the use of a synchronous tool. But, I will need to do more research to confirm this.
I talked with Jeff, the lead for the autoclave course project, and discussed using the topic for the project. He is very supportive of the idea. I think he feels the exposure would be great for the engineering groups.
I Discussed using the autoclave with Art, the Instructional Designer for the course. Art feels the course could be easily adapted for a web-based approach.

In reflection, I feel the initial stakeholder discussions are going okay. I do feel that I need to move to a greater level of detail in planning. It is time to clearly define the benefits and the details. I also need to meet with other stakeholders (POQ management) and gather their support. As usual, I need to be more assertive in moving the project forward.

In response these needs, I think the best action is to develop an initial draft plan (using microsoft project, powerpoint and word). I need to then distribute this plan where appropriate. I also need to meet with Cindy and Kim; both key stakeholders.

September 9, 2004 - Library and Project Focus

2004-09-04T20:16:00.000-07:00

I visited the Pepperdine library this week to gather research materials. Although I came away with some good books, the library orientation was invaluable. James the librarian was very helpful, and walked me through an entire web tour. I think the Virtcamp week was so overwhelming, that the short introduction to the library just wasn't enough. I now feel much more confident doing my research. I also learned that the Westlake campus also has a library - what great news! I was also reminded that Amgen also has a very reputable library. I was sort of embarrassed that I had forgotten about my companies own resources. Oh well!

After giving it more thought, I'm leaning toward an ARP topic on distributed learning. The problem itself might be defined as : A lack of knowledge and expertise shared across Amgen's sites in the area of technical training. The question is: Can distributed learning, using both synchronous and asynchronous learning tools, help distribute knowledge across all Amgen sites? I shared this idea with my project manager, and she thought the action is realistic.

I guess the next step to define further my problem and the question. I do feel good that I am making progress and look forward to the next steps.

August 30, 2004 Meeting & Research

2004-08-30T22:36:00.000-07:00

I met with my manager Paul today to discuss more of the project. The discussion was still very general, but I believe I have his full support for the project.

We talked about my primary ARP idea - applying distributed learning to our technical training. We agreed that the action associated with the project would involve converting an existing ILT to a distributed course. The course should represent a typical class - not too simple or too complex. I felt the need to suggest specific ideas, but was concerned that I was too early into the process.

I still uncomfortable with selling my ideas or taking a real assertive lead on this project. I guess I still feel out of my comfort zone still. I know that being more prepared and organized prior to meeting will help me feel more confident. Therefore, I will schedule a follow-up meeting tomorrow to discuss possible topics. I will prepare well in advance with a clear agenda and a set of ideas ready.

Since Bb was down tonight, I was able to continue to research for articles off the web. I found some good sources on distributed learning. The reading is beginning to build both my confidence and knowledge of the topic. For future searches, I hope to find more examples of distributed learning related to technical environments.

August 29, 2004 - Is It Personal?

2004-08-29T21:54:00.000-07:00

I find myself questioning my ARP ideas. Are they realistic in my organization? Are they real problems that need to be solved, or are they just areas of interest to me. Are my ideas personal enough?

My concerns started after our TI session this week. We were discussing some of Jason's ideas. What stuck with me was the discussion about the "personal" aspect our ARP. The APR is as much about changing the researcher as it is the organization or client. So to make it personal - it's as much about me as it is my work area.

So I ask the question - will implementing some type of distance learning program in my company impact me personally? I do see it solving organizational issues. But, how is this personal? Well, I do feel a tremendous need to apply technology to my workplace. This is a fear and barrier that I want to overcome. After all, this is why I applied to the OMET program in the first place.

I also looked at another possible focus for the ARP - the on boarding and support of the contractors that support my projects. The whole process of sharing information is very unstructured and inefficient. Maybe fostering a shared "communities of practice" for my staff would be beneficial. This would definitely impact me personally. The topic just doesn't seem as challenging. But is having a "challenge" the goal?

I know now that it will be reflection that is the key here. I need to better reflect on who I am in the organization and where I want to change and grow. I have a lot to think about.

August 22, 2004

2004-08-22T21:51:00.000-07:00

I met with both my old and new department managers to discuss further defining my field of action. Both managers were very supportive and were ready to provide numerous suggestions.

Before I met with them, I got some excellent advice from another manager and my project manager. Here is their advice:

Keep the project scope manageable. Shira mentioned that even the smallest research project can easily grow too quickly in scope.
Both suggested that I pick something closely related to my projects.
I should chose a topic of my interest, but it should be manageable.

When I met with Paul to discuss the project, I already had a good idea of my area of focus. There are several problems I would like to examine:

The expertise of our operations staff is not spread evenly among all sites. This means that a tremendous amount of resources are being used to train staff on sites lacking skilled workers.

Another related problem is the limited communication across sites. As a result, knowledge is not being shared on a consistent basis. When expertise is shared, it normally requires staff and trainers to travel. Obviously this can be costly and very inefficient.

I would like to research two possible actions to solve these challenges:

Develop distance learning that would be both synchronous and asynchronous. All training would be conducted online using a variety of tools.
Develop on-going communication and learning by developing learning communities within operations.

I will most likely limit the scope of the research to one job/discipline at Amgen.

The great thing is that Paul is very interested in applying distance learning within our areas.

Ready To Go!

2004-08-17T07:56:00.000-07:00

I am just beginning my ARP journey. My field of action is taking shape and my journal is ready. "Even a journey of a thousand miles begins with one step." Well this is a good first step!