Sunday, March 18, 2007

There is no mystery in Mystery Shoppers

Nobody wants to be thought of like Ernestine, Lili Tomlin's rude telephone operator! We have been trying to improve customer service in our hospital. Academic medical centers are often not great at helping patients navigate their way through their clinics, and we are hoping to set a higher standard at our place.

We have borrowed the concept of "mystery shoppers" from other service industries. We train people to pretend they are patients or patients' family members, and then we send them into a clinic to make observations and take notes. We also do this with our call centers, so we can see how our people serve the public on the telephone.

We then share the results with the chiefs, the clinic managers, and, of course, the front-line staff. As in the case of clinical improvements, we do not engage in the "blame game", but rather we use the shoppers' reports to offer helpful suggestions to people. Often, too, the problem is not with the front-line person, but there is some systemic problem behind the scenes that needs to be fixed.

Curious? OK, here are samples from two of our clinics. In the first, the Emergency Department after a very busy day, you can see areas for improvement. In the second, the more sedate Infectious Disease clinic, things look pretty good. (Excuse the stream-of-consciousness feel of the reports. We ask our shoppers to maintain a running commentary of what they see and hear.)

Remember, these take place in the waiting rooms -- not the patient care areas. These particular surveys are designed to review service quality, not the quality of the medical care offered in the exam rooms.

The Emergency Dept wait area is comfortable, well designed area with corner views of the Medical Center area. It is well lit with natural Department and fluorescent lighting. Area is modern but messy and dirty. On today’s visit, which occurred late in the day after the ED had been on diversion, there was litter strewn about, empty soda bottles on side tables and food crumbs all over the floor. Entryway is free of obstruction to passage. The first group of seating is reserved for patient triage. Security greeted me as soon as I entered and told me where to wait. The guard would not allow me close to the desk area unless I required nursing assistance. I wasn't sure how patients were actually checked-in, since the security guard primarily interacted with the patients who arrived and he asked them to have a seat. There is a large C shaped desk that seemed to be shared by security and nursing that was part of the wait area. Once I took a seat further back in the area I was not acknowledged again. Co-payment and referral signage were not posted. Patient Rights and Health Care Proxy information were available in various languages on a nice turnstile rack that needed restocking. There was no PRC information. Infection control information was posted throughout the area. One box of tissue was on a side table; no Calstat bottles. There is a vending machine in the back along with restrooms. Restrooms needed cleaning. General appearance of the waiting area was messy. At the check-in desk there were soda bottles and a coffee cup. Area was lacking in entertainment reading; there were a couple of old, torn magazines scattered around. Addresses were visible and there were no instructions for coping. Several informative brochures were neatly arranged on a long side table. Area had a plasma screen television which was on, tuned to nightly news. RN triage area is further along the large C shaped desk area and provides for privacy. Ten patients arrived within fifteen minutes of one another and overall the staff managed the patients efficiently through the triage area. Information regarding waits was not provided once in rear wait area. Staff was pleasant and courteous. Name tags were visible. Nursing personnel wore lab coats. At times personal conversations occurred between security and nursing but were not overheard. The security guard was chewing gum. I asked security (my only option) for directions to the cafeteria. He instructed me on how to proceed to the Farr Building then said -- "you’ll find it". My assessment of customer satisfaction in this clinic would be 3.0 -- good patient flow system but lacking in friendliness.


The Infectious Disease Clinic shares a good size waiting area with the Travel Clinic. Area is up to date, warming, inviting and comfortable. When first entering the French doors you see the water cooler and excess bottles. It does not pose an obstruction to passage. The reception assistants made eye contact with me immediately. I chose a seat in the corner as she inquired how they could be of assistance. There was no posted signage on the walls or framed on desks. Kleenex was on all side/reception tables. One Calstat dispenser was on the wall by the entrance area; no others visible in the area. Appearance of the waiting area was more than satisfactory -- clean, neat and orderly. Plenty of updated reading material for entertainment and infectious disease related. None of the magazines had stickers with copying instructions. Names were blacked out. Most of the magazines were addressed to MDs in the Lowry Building. There was no television in the waiting area, more than adequate seating and no clutter on the reception desk. A small bouquet of fresh flowers was in the room along with several healthy appearing plants. One of the staff members was caring for the plants and making pleasant small talk with those waiting. I would rate the area a strong 5 in the customer satisfaction area. There were 3-4 attendants behind the desk at all times. They spoke quietly, discreetly and professionally. None wore lab coats but were dressed in professional work attire. All had their name tags visible but I was unable to obtain names without further calling attention to myself. Twice 2 different attendants approached me in the corner and asked how they could be of assistance. Therefore, I was addressed/approached a total of 3 times in 22 minutes. One gentleman was seated several chairs next to me the entire time who was never approached. But he seemed content. One patient was identified by their first name but it was clear they had an established relationship. Physicians in the area wore lab coats, walked in the wait area to greet their patient, shook hands and lead them to the exam area. A physician approached one gentleman to offer an explanation of the wait after I was there approximately 15 minutes. As I was exiting the area I asked directions to the bathroom -- interrupting a work related phone conversation. The attendant was extremely pleasant, giving specific directions. As I was exiting the area a physician who heard my request offered additional instructions.

15 comments:

Elliott said...

Supplicant vs. invited guest?
I doubt it's explicit or conscious, but there is no doubt that it exists.

ID runs at a profit and ED loses money. Also, the safety factor is important. How many times per week are there police in the ED vs. how many times per year in ID? Finally, think about volume. All in all, I think you might be being unfair to the ED when you compare their budget per patient vs. ID's budget per patient.

Ask your janitorial staff to hit the ED waiting room every time the number of people loged in equals the number of people who entered the ID waiting room for the entire day. Assign the same number of desk people in the ED waiting room per patient as you do the ID and I imagine that things might improve considerably (although your budget might take a hit).

Finally, do you have an non-emergency clinic to divert to? This seems to work well in hospitals that have this option.

Paul Levy said...

Of course, you are right that there are lots of such differences between the two clinics. I was not trying to equate them by telling both of their stories -- nor trying to denigrate one nor extoll the other. In fact, I remain incredibly impressed by our ED folks, especially on those really busy days.

No, we don't have a non-emergency clinic to divert to.

The Doctor Job said...

That is a great idea. While some people's opinions will always be skewed negatively or positively, it can give you invaluable feedback that you'd never even have thought of.

Anonymous said...

I like the idea of a mystery shopper. My small town hospital redid their ER dept and several other depts after they merged with the other hospital in town. I moved into town right when the merger occurred and I was less than impressed with the hospital. Staff was rude, the place was a mess. My daughter has severe asthma, and the dr was telling that her 89 O2 sat level was ok. I immediately took her to her asthma specialist the next morning. The dr told me not to go back to the er there again. To call Children's and get the fellow on call and they could listen to her breathing over the phone and call in a rx or have her come into the er at children's (hour away).

Since then they have merged. They have sent out patient surveys. Asking about all aspects of care. I got one for a lap that was done. Two things that bothered me. One the OB ward did not call me to follow up on me after surgery and they are supposed to call after every surgery. Two, the anesthesiologist tried to say I was there for a tubal and I was there for a lap for endo. He kept saying your chart says. I said THAT IS NOT MY CHART. Finally my ob and the nurse came in. He said that she is here for a lap. The other girl there is here for a tubal. He said we are not switching, lol. I did report that. Because I was already anxious and just didn't need the additional anxiety of it.

The ER has gotten better now. Except, if I come in with a problem now, they contact my physician and let them know that I am there and they give him the results of all the tests, and then he agrees or disagrees with the course of treatment. They still have way too many personal conversations that patients can hear, but the care is getting better.

Anonymous said...

I wish Joslin would do this. They could really use a lot of improvement.

Ileana said...

I wonder what is the right way for us, patients, to send feedback like this? Is it even useful? Do you get so much feedback that you can't sort through it?

You would send something if you are very unhappy or very happy with the treatment you got, but if you are just a loyal patient (not angry, nor exited) that notices things that can be improved, what is there to do?

Craig Williams said...

This is an interesting post. Can you talk about your structure around the patient experience? For instance, who receives and is responsible for following up on these reports? How much organizational commitment is around this? Based on your post, I assume that folks at BIDMC understand that the patient experience is an important initiative there and that all levels of management are engaged? Do you feel like it is ingrained in the culture or are you struggling to get people interest in this?

jodim said...

Thank you for your support of healthcare mystery shopping. I own a medical mystery shopping service with focus on the private practitioners, managed care groups and extended care facilities. Not only is it a means to monitor employee performance and assure compliance, it will hopefully improve the patient’s emotional experience. Patients seek quality clinical care when ill but also want (and need) to be treated with kindness and compassion. I can personally attest to being made to feel like a nuisance rather than welcome, more often than not. Many healthcare providers are often unaware of the patient’s perception and this concept can truly be an eye opener. As a business owner and a patient, I’m glad to see an increase in interest.

Paul Levy said...

To Craig,

"I assume that folks at BIDMC understand that the patient experience is an important initiative there and that all levels of management are engaged? Do you feel like it is ingrained in the culture or are you struggling to get people interest in this?"

This is indeed one of our top priorities, and people throughout the organization are involved. I can't give the whole story here, but rest assured that results are promulgated, discussed, and acted upon.

Anonymous said...

I recently(end of February) had a CT scan done at BI. I was a bit early and parked at Shapiro. Shapiro is a lovely building, airy, spacious, clean with high ceilings. I followed the yellow brick road (i.e. the marked path) to the 3rd floor Rabb where my appointment was. Following the path worked pretty well. It got me to Rabb ok, but once I was in Rabb I was lost. Not only couldn't I figure out where the CT section was, I couldn't even figure out what floor I was on. Once I did find the third floor, the signs again led me to the right place. HOwever, Rabb was hideous, low ceilings that looked as if they were about to fall down, walls that closed in on you.
Though clean, it looked very ratty. The CT check in area was tolerable, though the bathroom didn't lock. But the waiting room!!! The waiting room was small and had maybe 12-15 chairs, and with four people was totally full. Again, it felt very grimy. A person in a wheelchair took up a lot of space. There was no privacy for a woman when the nurse came in to talk to her about her scan being delayed. The changing rooms were unidentifiable as such - in fact - I thought they were closets until the nurse told me to go change there. However, everyone was extremely nice; all the staff from the check in person to the technician. And when I left, I walked slowly back to Shapiro trying to figure out what it was about Rabb that made me hate to be their. I walked back looking at the ceiling heights and at the walls - i must have looked lost - because 2 or 3 people actually stopped and asked if they could help me find my way!!! Extremely nice staff. So, yes - it takes money to refurbish buildings, but maybe starting by painting the walls, putting up some paintings (like there are in Shapiro), enlarging the CT waiting room?

eeka said...

That's really great. I've actually had a lot of ideas about similar research, but it seems that you can only go so far with this kind of research. As far as I know, it's been deemed unethical to have a researcher fake an illness (you probably know about the pseudopatient studies done in inpatient psych facilities in the 1970s -- if you don't, I'll explain). The field has decided that having a researcher pretend to need care might skew data (both actual numbers and the providers' own perception of how various illnesses play out). Also, it's generally frowned upon to waste resources like this.

All of which is understandable, but is really too bad, because there are several scenarios I'd love to run -- English-speaking and non-English speaking patient running otherwise the exact same scenario, and any number of other things.

What exactly are you allowed to do, other than have people loiter in the waiting room? You can't actually have people register or be triaged, can you? This is really intriguing...

Anonymous said...

I would be curious to know what percentage of BID staff give their name when answering the phone. In my experience as a BID RN, it's ~25%. (one person refused to give me her name when I asked!)Residents answer pages with "I'm answering for purple surgery," but don't give their name until asked. Many areas of phone etiquette need to be addressed here.

Healthcare Mystery Shopping said...

I was extremely excited to see the AMA's recommendation on
healthcare mystery shopping The Ethical & Judicial Affairs report concluded that "Physicians have an ethical responsibility to engage in activities that contribute to continual improvements in patient care. One method for promoting such quality improvement is through the use of secret shopper “patients” who have been appropriately trained to provide feedback about physician performance in the clinical setting."

CoryJ said...

Great Blog. I own a mystery shopping company in Omaha, NE that is now offering servies to healthcare providers. Our clients have found the information very beneficial in knowing if everyone is doing everything they should be and now have an unbiased tool to reward their stars and identify problems that exist.

One crucial thing to keep in mind, with all the privacy regulations now in place, it is essential that anyone performing this service not reveal ANY patient information.

Stacy said...

This was a great article that provides needed support for medical mystery shopping! I work for a company that specializes in healthcare and one service we offer is mystery shopping. This was not always the case, but came about as a result of so many clients requesting it. The results are rewarding and have helped to turn entire departments around just by seeing an outside viewpoint. Seeing the experience of a doctors visit through the patients eyes is invaluable and now common place in many healthcare facilities. So glad to see so many healthcare executives getting more involved in their patients experience and helping their staff to better understand what is happening around them! This is beneficial for everyone involved. After all, we are all patients at some point.