Two topics: Patient Care Systems and Greek-language search engines

 Double header today: Computerized Patient Care systems and Greek-language search engine evaluations…

Investigation on the Use of Computerized Patient Care Documentation

Preliminary Results presented by Kenneth Hammond

Kenneth et al. evaluated an existing medical records system using a cognitive work analysis framework. This was a large and mature system: 1 billion documents and 7 million patients. They sought the perspectives of its administrative users and clinical users (nurses and practictioners).

Methodology:

The health care environment involves high stakes, teamwork (nurses, practictioners, etc..), and complex workflows. The researchers sought to explore the space from a Cognitive Work Analysis perspective. They sought to understand the goals and constraints of the actors. They focused on the processes of the user’s behavior w.r.t. to the information and the tools they use. This was done with 30 minute interviews with 11 different staff and “think-alound” observations.

An excerpt of the findings:

1. There is an explosion of notes. 1 patient has notes by the doctor, nurses, and everyone who has something to do with the patient.

2. For outpatient situations, practitioners only see their patients over long stretches of absence. They depend on notes to refamiliarize themselves with the patients.

3. The notes that nurses write.
a. Are unread by practitioners and unread by nurses.
b. Are written for duty and as a record: “If you didn’t write it down, it didn’t happen”.
c. Are frustrating for nurses to write because their job is to tend to the patient. Time spent away from the patient doing data entry feels wasteful.

4. Nurses “bootlegged” the system to leave messages for one another.

5. There is a conflict between the wish to narrate and the desire to be precise: free-form input versus strict forms.

There were benefits to the system compared to its predecessor: greater access to data, legibility, accessibility, context-appropriate alerts (like warnings about drug interaction).

In response, they produced alternate designs addressing design flaws and demonstrated them. (e.g. They addressed interface issues such as typing in the date as plain text instead of clicking with a mouse; added sorting and grouping features, etc…). These changes were positively received by the users.

Questions:

Will you be using a prototyping approach in the future?
Only light work to validate some of our concepts. We want to produce principles and recommendations for the upcoming system redesign.

Did you look at the work they were doing separate from particular system features?
We observed them throughout the day as they did their tasks. Then, we investigated system-specific features. In the future, we will focus on the document work, using focus groups to try to classify the different tasks that they are trying to achieve.

An Evaluation of How Search Engines Respond to Greek Language Queries

Efthimis N. Efthimiadis

Internet usage by language shift has shifted from English to Non-English: 70.5% is non-English. With regards to the the Greek language, Efti seeks to: assess the effectiveness of search engines in satisfying user requests, evaluate how well search engines respond to Greek language queries, and compare the results of greek and global search engines.

About the Greek language:

There exist different versions of the language: Classical and Modern Greek. Modern Greek itself has different types. It has its own character set. The language includes accents, in particular, some of which have a effect on the pronounceation of a set of characters. Furthermore, the Web has many documents that use different styles of transliteration of greek (e.g. Greeklish).

Methodology

They chose 8 standard english engines (A9, altavista, google, MSN live, etc…) and 8 greek search engines. They selected a type of query to test: “Navigational queries” (from Broder’s classification of query types). Navigational queries are where the search engine is used to find a website that the user knows already. For example, to find the website for SouthWest airlines. They chose these queries because it would be easier to evaluate the results without resorting to complicated relevance judgements. They chose ~300 websites meant to be found (217 greek, 82 english) (governmental departments, travel agencies, colleges, banks, etc…) and then constructed queries: original greek names and non-Greek equivalents (if available). For each query, they retrieved the top ten results, and
use these results with a scoring metric to produce an accuracy score.

Results

From a qualitative perspective, Greek search engines usually handle the accents better (which can completely change the meaning of a query).

Overall performance of all the engines: Google (73% success rate), Altavista (71%), Yahoo(63%): topped the list. #4 is Trinity #7 (~50%) Visto (both greek). Note that a 50% success rate is considered pretty poor.

With just Greek queries, Altavista (72%) and Google (70%) still ranked best and the greek engines bombed: all under 50%.

Conclusions:

There is still much work to be done for everyone.

Discussion:

Q? What are the design implications for this study:
Efti: We need better processing of language models for foreign languages: For Greek, process and understand the greek.
Q: However, would the text processing overhead hurt or help
Google’s existing algorithm?

Efti: Also, another need is better crawling of the greek webspace. Almost half of the 70,000 greek sites (collected by Efti a few years ago) were not indexed by search engines.

Q1: Do pages in Greek lack the meta-data that helps search engines? That might be another issue. the act of adding metadata to a site might be affected by cultural aspects, rather than the properties language itself. For example,
there are findings that show that US pages use more metadata tags than other language pages.
Q2: How significant is this for search engines: do they use them to index anymore?
Efti: On a related note, HTML editors are often Latin character-based. This changes how Greek pages are authored and, therefore, indexed.
Q1: For example, often, people use jpegs of text instead of fiddling with character-sets.
Q3: Supposedly, these are issues are all addressed by Google’s methodology, which is content agnostic: relying on page rank and links instead.

Q: I disagree with Efti’s expectation that two queries with only different accents (and the same meaning) should return the same list of results. The different results that Google returns are a feature, because the different accent can imply a different intent by the searcher.
Efti: This is a debatable point.
Note: We compared this to the difference between searching for “cat” and “cats” versus the difference between searching for “cat” and “chat”. It turns out that Google returns different results for all variants. Efti appeared to argue that the semantic difference should be nil for the Greek example and that his tests show too much of a difference between the two test-queries.