Wednesday, October 20, 2021

Evaluation and Review--A scientist's life


L-R: Tom, Doug Rumble, Marilyn, Joe Boyd, 1990


A White Dude from the ‘50s


Tom Hoering walked to the Geophysical Lab on Upton Street every morning, arriving on campus by about 8:00 am. In winter he sported a long, beige trench coat and a woolen pork pie hat, gloves, and if it was snowy or icy, an old-fashioned pair of black galoshes that slipped over his shoes. Typically he wore a white shirt, an older, slightly rumpled tie, and a pair of trousers, usually khakis from LL Beans. In spring, he donned a sport coat and wore it throughout the day, unless he was glassblowing or using chemical solvents. In summer, he lost the tie, wearing a thinner, short-sleeved shirt.


He worked throughout the morning, started by spending an hour or so writing on a typewriter perched to the left of his old oak desk. Then, he’d saunter into his lab, turn on instruments, gather his “tools” for the day and make measurements.


If Tom’s phone rang, he’d shout, “A ringing telephone is a request, not a demand!” and keep working.


By noon, he’d meet up with his colleagues, Felix Chayes and Hat Yoder, then the threesome would walk down to Connecticut Avenue to eat lunch at the Hot Shoppes. Some days, he’d have a martini lunch, afterwards putting his feet up on his desk and taking a snooze for 30 minutes or so. He’d head back into the lab where he often remained until his day ended at 5:30 pm. Sometimes, but not often, he’d pack up a leather brief case with some papers to take home. In the evening, he enjoyed cracking open a new issue of Geochimica et Cosmochima Acta with its orange and white cover and reading about the latest scientific discoveries in geochemistry.


Tom fishing on the R/V Longhorn, 1974

On Saturdays, he strolled into the Lab around 9:00 am, usually in more worn out khakis, older shoes, and an open front shirt. He’d tap some memos out on his typewriter, check out his lab, walk around the building to make sure no water was leaking, or any other not-to-unusual things were happening in the Lab. By lunchtime, he’d head back home.


Yearly, Tom worked seriously on one or two projects typically culminating in a Carnegie Annual Report article in June. Often, he published his work only in this informal, yet widely read, medium. If he was into something “hot,” he’d work on it until he was truly satisfied, then would write a proper peer-reviewed manuscript. He usually mentored one postdoc at a time, as well as counsel a parade of scientists both young and old who wanted his advice. Tom loved giving advice and cared deeply about the people in his scientific sphere.


On the home front, Tom’s wife Martha took care of their two children, the house, did the shopping and cooking, washed his clothes and took his suits to the dry cleaners. He had very few home life things he needed to take care of in his early years. [This situation changed drastically when Tom was in his late fifties. Then, he took on the health care of Martha, learned to cook and garden.]


Tom was truly a White Dude of the 1950s. Over his career, he published about 60 papers in journals along with another 60 or so in the Carnegie’s Annual Report of the Director of the Geophysical Laboratory—not many by today’s standards. He wrote letters of recommendation, probably reviewed 6 papers and a few grants per year, gave a seminar or two, and went to one national meeting annually.


Treibs award ceremony, 1987

But he was enormously successful and influential, because the work that he did do was creative and thorough. The standards he set for himself—analytical rigor and novel hypotheses—he looked for in others. He didn’t “count” papers or demand outreach or service. He viewed an early career scientist with an eye to figure out whether that person came up with original ideas that could be tested. He looked for analytical prowess. He recognized a good speaker and a good writer. He also recognized bull shitters, slackers, and fakes. Knowing this, he attracted competent people.


Kate Freeman's Speech for Tom's Treibs Medal Ceremony, 1987

Tom’s counterpoint on the west coast was Sam Epstein, professor at Caltech. Sam’s influence in geochemistry is equally influential, if not a tad greater, as Tom Hoering’s. Epstein published about 130 papers in his career, often with one of his bright grad students as coauthor. The two men resembled each other in appearance with curly grey hair, short stature, and slightly formal manner. Both knew their stuff in the lab, could recognize talent, and were very creative people.

Sam Epstein, Caltech


Today it is popular for some entering the early stages of a scientific career to think that White Dudes from the ‘50s made up the criteria for which people are judged today. I don’t think this is necessarily so.


Communication until the mid-1980s happened by snail mail—letters pounded out on a typewriter or written by hand. Telephone calls were expensive. Personal computers were uncommon. Secretaries typed your manuscripts. Graphics were done by hand or sent out to a professional drafts person. To travel, people used travel agents and held paper tickets. To read the literature, you needed to physically go into the Library, take a book off the shelf, and then Xerox the paper you’d want to read later. Life was slower, much less complicated.


How are scientists evaluated today—1950s ethics or something more relevant?


For the past two years, I served on AGU’s Macelwane awards committee, reading and evaluating 60+ nominees each year, then having to choose only 5 to get the award. These nominees, all within 10 years of earning their PhDs, have remarkable records of accomplishment! Many of them have published 60 to 90 papers, have brought in millions in grant funding, established new programs, often before becoming full professors. This simply phenomenal rate of accomplishment isn’t based on White Dudes from the ‘50s. 


I’m often asked to provide an assessment of a professor for tenure and promotion. Some universities provide a candidate’s written statement 20-25 pages long, not including a separate CV. University “bean counting” makes them list every talk, seminar, paper review, and student evaluation so that people like me can write an assessment. The sheer amount of tasks that early career earth scientists must undertake to be “successful” is numbing. Based on feedback from colleagues at this point in time, it can also lead to burnout.


Where did the metrics for scientist’s reviews come from? How often are we evaluated and reviewed in our careers? How often do we evaluate and review others?


Here’s an estimate.


I published about 200+ papers in my career. For each paper, an editor judged it 3 times, and three reviewers read and commented on it twice for a total of 9 evaluations/paper and 1,800 evaluations for the body of work. For the people who actually read the papers, maybe 10 formed an opinion, adding another 2,000 “reviews” to the mix.


Informality years ago...

I submitted maybe 60+ grant proposals during my career. At NSF, for example, a program officer evaluated it three times, three outside peers provided reviews, as well as 8 panel members: 840 reviews in total.


I gave about 2 seminars per year and 2 talks or posters at conferences to audiences of about 50 people each time. Say 10 of them formed an opinion—in 45 years, that’s 1620 informal reviews!


I came late to classroom teaching--600 students total in 8 years: another 600 evaluations.


In the collaborations along the way, I worked on over 200 projects with at least 3 colleagues that commented on all aspects of the work multiple times: 1,800 evaluations of approaches, methods, and data. Add to that coauthor reviews of manuscripts: another 600 reviews and edits.


That’s 9260 evaluations, conservatively, about 4 per week for 45 years.


Here’s an estimate of how many scientists I have reviewed or evaluated.


I probably review 6 manuscripts per year: often twice: 405 reviews.


Thirty some postdocs and 40 some grad students were evaluated informally probably three times a year for 3 years each: 630 evaluations.


In classroom teaching, 600 students took 3 exams and wrote one paper: 2,400 evaluations.


I review maybe 5 grant proposals per year (for 30 years): 150 reviews.


I attended about 30 seminars per year and probably listened to, and paid attention to, about 25 talks per year for 45 years: 1825 informal evaluations.


Tenure and promotion reviews are much less frequent, maybe three per year for 20 years: 60 detailed assessments.


Editing manuscripts I coauthor takes time and I work on them until they seem as good as we can make them. 200 manuscripts edited 4 times each: 800 reviews.


At the University of California, I reviewed faculty “files” for merit and promotion increases, usually 7 per year for 7 years: 49 university assessments.


6319 formal and informal assessments, reviews, and evaluations. For 40 years, that’s about 3 per week, for 50 weeks every year.


The Yin and Yang of it all is clear. People judge you; you judge them.


Scientists expect they’ll be reviewed, but most of us don’t like it, particularly if the reviews of ourselves are negative. Here’s what I’ve learned.


No one likes a review that starts like this--ever.

1. Be thankful for any editorial improvements that your colleagues and anonymous reviewers make on your manuscripts. In general, if someone tells you your sentence is unclear, it probably is. Accept their help. I advise against “let’s submit it now and let the reviewers make it better.” Always submit your best work.


2. Grants are panned for different reasons—not enough funds, greater scrutiny when money is involved, and protecting turf. When your proposal is nixed, reflect. Could it have been clearer? Is it your best idea? It’s easy to think your greatest critics are the people who have given negative reviews. Based on my time at NSF, I learned that who you thought panned you, wasn’t usually the case. Re-submit at least once, if not twice, if this is one of your best ideas.


3. When you give a talk or seminar, practice in front of colleagues and your supervisor during your early years. Give yourself enough time to revise your presentation. For more seasoned folks, prepare in advance, not just the day of your talk! Make the presentation shorter, not longer. Your audience will be pleased that you don’t talk longer than your allotted time.


4. Poll students mid-semester to find out how you can improve as a teacher. Make an effort to see what helps them succeed, while having to keep your own ego unbruised. When you do get student evaluations, take a breath. Many of my own evaluations and those of my colleagues contain criticism about accents, clothing, appearance, style, and computer skills. What irked me the most was when students wrote that I was “unprepared.” Keep a talk about your most intriguing research in your “pocket” when that might happen. I have found students love to be able to sit back and be entertained by something as cool as your personal research.


5. For that big tenure review, I’ve seen two different responses by folks at this stressful time: cool and prepared vs. nervous and worried. Some people are by nature nervous and worried. If you are one, plan early and start making personal connections with people who might be asked to write an assessment of you. Is this outright schmoozing? You bet. More likely you’ll make a valuable scientific connection that might serve you both for years.


Steady progress is best…a paper a year may not seem like a lot, but by six years, you’ll have six papers. Additional manuscripts from colleagues and your students should supplement your own first author contributions.


While it’s good to get your self “out there,” too much outreach, too many talks, too much service can be exhausting and actually dilute what is most important: are you independent and can you “drive your own bus”?


Cool and prepared folks have kept a steady but not overwhelming pace. They publish some of their postdoc work to keep the stream of science coming while establishing their labs, find grad students and begin teaching. Personally, speaking up for your self and others, contributing to faculty discussions, and being a positive person helps. Having a healthy work-life balance is, in my opinion, somehow helpful in the process.


Back to the question of where did the extreme review mantra come from?


I think it’s exacerbated by the fast pace of the internet, email, and even social media. Your citations are no longer difficult to track, but can be easily accessed via Google Scholar. Now, people can see what you’ve been publishing instantly and how well its been received.


Students no longer fill out evaluations by hand in class, but can do so at home online.


When people give a popular talk or publish a good paper, it can instantly be sent out to the world on social media. [I do this as well.]


There are simply more early-career scientists these days—and greater competition for coveted jobs.


Finally, I’ll add that it might be that Older White Dudes of the ‘70s and ‘80s may be the department chairs, deans, and directors who are judging your work. The phrase “Back in my day…” is all too prevalent sometimes. Back in my day, the world was simpler. Today it is more multi-faceted and complex.


Now, many universities have added accomplishments in diversity, equity, and inclusion as another aspect of academic evaluations. These added criteria are important, but I’ve noticed that we don’t yet have metrics for properly evaluating them—in either BIPOC scientists or Older White Dudes. Eventually, we’d all like to reach fairness and respect for everyone. In my work as College Equity Advisor at UC Riverside, I’ve seen departments that “get it,” while others are struggling to make this work for them. COVID has injected another layer into the mix with work stoppages for parents, early career job seekers, and those with health concerns.


I know this is a controversial topic and indeed a tricky time. I hope that we’ll come out the other side enlightened and committed to fair, thoughtful, and even helpful, reviews.




Rounding Third Base and Heading Home

Cards from Franny and Flowers the Rumbles   My daughter Dana is marrying George Goryan on June 25 at our home in Mariposa...