James, I am sure this is important stuff that I want to know. But I will never listen to a podcast as is true of many readers here. Wrong format for us and fundamentally far to inefficient. Most sites to which I have whined have found auto-transcribers for their podcasts which, while not perfect, are good enough. They generally post the transcription with the podcast which is wonderful for all communities of learners, visual or aural.
I hope you can do this. Many of your peers on Substack do this already and it is most appreciated.
Thanks for the good things you are doing. I cannot wait to read this particular dialog, frankly.
Abaluck has a really good point at 17:50, "Well, hold on hold on hold on holelelelele."
His apologia for the "masks were working" headline is pure semantic illogic. If a study finds that trucks can carry logs but cars not so much, then the headline "automobiles could carry logs all along!" conveys a meaning (all automobiles can carry logs) that is contradicted by the study. "Some" ≠ "All," and "All" is the default implication of a sentence when a modifier is not added to the subject.
Sad and embarrassing that he can't see how flimsy his own counter-argument is here. But spend enough time in a cult and you forget what real debate consists of.
I was visited by a stranger with family in the FDA a few weeks ago, and the conversation went the same way. If I said severe outcomes from infection were rare, I was told to understand that that is not how I "have to think of it." The same way Abaluck thinks you are "misunderstanding" his study, as in thinking about it in a way different than how he thinks you "have to think about it."
It's a train wreck, really - hard to keep watching. I hope he gets around to specifying what the denominator was for the seropositive rate.
43:00 He seems evasive on the issue I wanted the most clarification on. But the paper has been updated with more detail since I first read it. It seems that "Baseline symptomatic seropositivity" was the positivity rate among symptom+ responders on the week 0 survey. But then is it the same people being tested at follow-up, and what kind of valid conclusion can be made from that comparison anyway? It's still the messiest part of the paper and the only one that "supports" any conclusions about efficacy vs SARS-CoV-2 as opposed to efficacy against "saying you feel sick on a survey."
If the "baseline" is a symptom+ group then the difference in follow-up should be huge regardless of the intervention, because most symptom+ will be pre-IgG-seroconversion. So it doesn't really matter if the treatment made a difference here because the signal is way too weak. This was my guess before the video and remains my guess now. There were very low case rates in Bangladesh during the study period (rolling from November 2020 to March 2021) so they were measuring a whole lot of nothing.
32:30 Here he's trying to have+eat his cake. A clear association between effect and treatment wouldn't rule out an indirect effect (i.e. from behavior changes). But he backs off before tripping up.
33:30 "Particles in the universe" lol. One obvious possible non-controlled confounder is listed in the study text, "We also do not yet have data on distance to nearby city." Matching was based on case rates in June/July 2020 and wouldn't ensure equal case rates in Nov-March, and wouldn't address changes in seropositivity driven by work migration anyway. The number of seasonal urban workers returning to or leaving villages could have driven differences in seropositivity by adding / depleting donors who got infected in Dhaka / Chittagong (where previously-reported seropositivity rates were very high). This isn't even necessarily distance-based.
I think Jason did a good job of defending his position. My takeaway was that there was some, but little benefit to masking, but it was hard to tell the degree because the controls were not as aggressive as desirable. Sounds like more studies are needed. That said, his call for the less knowledgeable to accept the consensus belied his remark that he did not take either side in the argument and simply sought the truth. I suspect he was biased going in and there is a good likelihood that skewed his work in interpreting some of the study areas with weaker controls.
This interview was challenging to listen to. There was so much interrupting and talking over that it wasted time and took away from the value of the debate.
I listened to it while driving, otherwise would have given up. I need to go to the actual study now to see what was being debated. I did learn a lot though. As always thank you James.
This showed how easy ti can be to misrepresent a study. So many factors and considerations go into building a viable study. Dr. Abaluck's study was a huge under taking and is to be commended for it. But his defensive posture made it difficult to listen to even if he was making a valid point. This suggests he had some kind of preconceived notion about how it would go down. Had he let his guard down he would have made his points more effectively to layman's like me because he's clearly competent. Still, he deserves credit coming on to discuss it. We need more of these debates.
That all being said. I'm not convinced masks work.
When I was a stock broker I'd hear all sorts of reasons and explanations form a company justifying their quarterly results. Or read about why a company posted a result but we needed to read the details. For sure, that was at times necessary but at the end of the day I asked a simple question, despite all the qualifications, do you make money? Are you profitable? Yes? No? Science has p-value, investments P/E multiple. And like Abaluck expressed not being a fan of p-value, there were investors who took P/E with a grain of salt or at least cautioned interpreting data based on it alone.
I apply the same thing here with the masks. I note that it's the rule and not the exception that data shows masks have marginal benefits. Yet, we always have to hear about caveats. Bottom line is do they work or not?
As for my impression of the types of masks, cloth masks are akin to t-shirts around our mouths. Useless. So enough of this cloth nonsense. As for surgical masks, we've had decades worth of studies - in both hospital and community settings - that showed they're ineffective. The talk about N-95 is also somewhat misplaced because they're simply not feasible in community settings. I wore those in construction and you'd better change them every coupe of hours....
I looked up Dr. Abaluck and see he is a professor of economics. Economics is the art of obfuscation and getting any data to say what you want or exclude it as an externality or some other construct useful for deception. It is not a science just as political science is not a science. I find it preposterous that anyone could possibly think they can barge into a third world country and get people to conduct some kind of behavior for the purposes of your study and then measure accurately the result of that behavior especially for something as nebulous as covid prevalence measuring. I really wonder how many people wore those masks as they were trained to do (surely they were trained just like medical professionals get trained in ppe usage). Did they make sure they fit well? Remember all it takes is openings greater than 3.2% of total mask area and a mask filters nothing. (https://www.tandfonline.com/doi/full/10.1080/02786826.2020.1817846) His vaunted surgical mask never fits well with openings all around the mask, just look at people wearing them(see the interviews with industrial hygienists). So we have professionals (Industrial Hygienists) whose job is the protection of people from airborne hazard and has a mature and stable base of principles and practices for achieving this. They say cloth masks don't work for viral particles, they say surgical masks don't work either. Now we have an economist who gets a bunch of people in Bangladesh to wear masks around and measures covid rates and sees if he can find a correlation. I have an even better one to study which is analogous to masking. The State of Utah has one of the lowest rates of Covid deaths per million in the country. Most people in Utah are Mormons, Mormons wear a particular garment as a part of practicing their religion. They believe that garment protects them and I am not here to say they are wrong or demean their religion. A mask is also a garment that people wear because they think it protects them even though the people whose job it is to figure out how to protect people from breathing in bad things say they don't. So, if some other economist out there decided that he wanted to prove the Mormon garment absolutely does protect from Covid I'm sure he could; just as Dr. Abaluck went to Bangladesh and got a bunch of people to wear masks and purported to be able to glean some sketchy data saying it protected them from Covid. Bottom line, if you want some tobacco science done get an economist to do it, they are so good at getting any data to fit any theory they might come up with. A better debate would have been between Industrial Hygienist Steve Petty and the co-authors on the study who Dr Abaluck referred to as "knowing more than you about masks". Then we could get some meat into the debate, does a given mask filter out virus? Can you get people to wear them properly for long periods of time given how uncomfortable it is?
Of course you can do simplistic mechanical studies (cited by Dr. Abaluck of coughing into a petri dish masked and unmasked) that don't really study how a mask is used but that is what economists are good at citing as externalities the fact that users don't follow their strict models and so are excluded. The fact is most people wear masks as clothing and are just complying and are not going to go to the trouble of wearing these things 'properly'. They are constantly messing around with them because they are uncomfortable and so their hands are constantly contaminated as is everything they touch. This is especially true of children.
When it comes right down to it, it is everyone's right to breathe freely without obstruction. The process of respiration is not defecation. This idea of required face garment is extreme and brutal and those who push it are despots.
My takeaways are pretty straightforward. I have very little expertise with statistics or designing/executing/evaluating scientific studies. That disclaimer aside, it seems to me that:
1) Your point regarding the language used to characterize the results of the study in the media are well-founded. The headlines you chose were actually the somewhat less bombastic and triumphant examples that were paraded about. Words have meaning and too often have the impact of distorting perception—the gymnastics Abaluck went through in qualifying his support of the headlines you cited were very revealing.
2) The primary findings of the study seem relatively weak, especially in light of ‘symptoms’ and the huge overlap between Covid-19 and other relatively common conditions. The applicability of this study’s findings to the Covid pandemic—specifically—doesn’t seem strongly supported, especially as it relates to the question of whether or not masks reduce infection or transmission of SARS-CoV2.
3) The social aspects, especially cultural factors, seem understudied. The specifics of Bangladeshi culture, norms, behaviors, customs, etc., are only vaguely understood. Given that masking is such a socially-entwined intervention, it begs the question of extent of applicability of this study to other cultures (the US, for example). This isn’t a question that is adequately addressed, but would seem to be fairly pivotal.
Though it was a long debate, I found it very worthwhile to watch in its entirety. You, Dr. Jack, did a great job of staying cool as a cucumber and patiently (persistently) teasing out the dialogue despite his initial defensive posture. I think there was a kind of (rickety) bridge built here, which is encouraging. Thank you!
None of the masks studied are for PATHOGEN PROTECTION. I think that is all we have to say here IMO. I would ask Dr. Abaluck: were any of the masks used in the study actually graded for pathogens? The answer is simply: NO. The rest is blah blah blah. The petri dish??? Isn't that bacteria???? He mentions at the beginning the cost-benefit ratio of masking. Now, I didn't read the study, but intend to. My question is, did they actually take into account reduced immune function etc etc from masking? Was it true cost-benefit analysis? If so, what affect did this have on findings? My guess is that they didn't and it does. So again, researching for just the spread of coV-2 and not reduced health outcomes....hmmm....reminds me of another debate.
Searching out this study showed me the clear bias in the science world as compared to real critical thinking. If this study only showed a 1% decrease in symptomatic cases between cohort, I find that result a nothing burger. How are the other percentages calculated....another question for Dr. Jack.
"The proportion of individuals with COVID-like symptoms was 7.62% (N=13,273) in the
intervention arm and 8.62% (N=13,893) in the control arm. Blood samples were collected
from N=10,952 consenting, symptomatic individuals"
So 10,952 were tested......from?? and of 300,000 individuals for 5 months, only 10,952 tested positive?? Were there other treatments administered? (HCQ, Ivermectin etc). Also, how many were already immune? I have more questions than answers. Another bogus study.
The only way they'll ever really be able to tell if any kind of mask has any effect on stopping transmission is to do human challenge studies where different groups of people wearing different types of masks for different lengths of time are purposely exposed to a respiratory virus (as I explain in this blog post: https://beyondspin.wordpress.com/2022/02/08/to-mask-or-not-to-mask-that-really-isnt-the-question-or-necessarily-the-solution ) . For those types of studies, you have to set aside ethical constraints....which obviously is very problematic. Otherwise all these studies, that wait for people to get infected, will always have a myriad of confounders no matter how much the researchers believe that they'e controlling for various variables. The aRR on the particular study you two are debating about isn't the slightest bit compelling, and other meta-analysis of mask studies showing rRR's of much higher rates (53%) have very weak confounded aRR's of around 0.5%. So again, not the least bit compelling. So even these RCT's are pretty much bull shiet.
If I'm getting this right, Dr. Abaluck is saying that the likelihood of any confounder in the study occurring is so low that it is pointless to consider it. Such as where another factor, like last names ends up correlating with the positive mask outcomes nearly perfectly. What you're saying is that a confounder occurring ( whatever it may be ) is far more likely than is usually assumed.
So there could be something else along with mask wearing that lead to the appearance of a statistically significant result for surgical masks. Last name ( genetics ), change in behavior of either the wearer, or the people they interact with, etc. Abaluck is saying there is virtually no chance this happened with enough overlap to the results to make it worthwhile to consider. Would that be a fair summary?
I personally find it very hard to believe that this study had the proper controls in place to make any result meaningful. There are far too many other factors involved. You would need to have both groups be so similar in nearly every aspect while at the same time conducting the study in the real world as to make the ability to do the study itself basically impossible.
James, I am sure this is important stuff that I want to know. But I will never listen to a podcast as is true of many readers here. Wrong format for us and fundamentally far to inefficient. Most sites to which I have whined have found auto-transcribers for their podcasts which, while not perfect, are good enough. They generally post the transcription with the podcast which is wonderful for all communities of learners, visual or aural.
I hope you can do this. Many of your peers on Substack do this already and it is most appreciated.
Thanks for the good things you are doing. I cannot wait to read this particular dialog, frankly.
Abaluck has a really good point at 17:50, "Well, hold on hold on hold on holelelelele."
His apologia for the "masks were working" headline is pure semantic illogic. If a study finds that trucks can carry logs but cars not so much, then the headline "automobiles could carry logs all along!" conveys a meaning (all automobiles can carry logs) that is contradicted by the study. "Some" ≠ "All," and "All" is the default implication of a sentence when a modifier is not added to the subject.
Sad and embarrassing that he can't see how flimsy his own counter-argument is here. But spend enough time in a cult and you forget what real debate consists of.
I was visited by a stranger with family in the FDA a few weeks ago, and the conversation went the same way. If I said severe outcomes from infection were rare, I was told to understand that that is not how I "have to think of it." The same way Abaluck thinks you are "misunderstanding" his study, as in thinking about it in a way different than how he thinks you "have to think about it."
It's a train wreck, really - hard to keep watching. I hope he gets around to specifying what the denominator was for the seropositive rate.
Dr Abaluck's "science" is summed up perfectly, by him, at 06:15. "uuuuummmmmmyeah".
The study shows that mask do virtually nothing for this application, as have virtually ALL studies for, um, several decades. Period.
43:00 He seems evasive on the issue I wanted the most clarification on. But the paper has been updated with more detail since I first read it. It seems that "Baseline symptomatic seropositivity" was the positivity rate among symptom+ responders on the week 0 survey. But then is it the same people being tested at follow-up, and what kind of valid conclusion can be made from that comparison anyway? It's still the messiest part of the paper and the only one that "supports" any conclusions about efficacy vs SARS-CoV-2 as opposed to efficacy against "saying you feel sick on a survey."
If the "baseline" is a symptom+ group then the difference in follow-up should be huge regardless of the intervention, because most symptom+ will be pre-IgG-seroconversion. So it doesn't really matter if the treatment made a difference here because the signal is way too weak. This was my guess before the video and remains my guess now. There were very low case rates in Bangladesh during the study period (rolling from November 2020 to March 2021) so they were measuring a whole lot of nothing.
32:30 Here he's trying to have+eat his cake. A clear association between effect and treatment wouldn't rule out an indirect effect (i.e. from behavior changes). But he backs off before tripping up.
33:30 "Particles in the universe" lol. One obvious possible non-controlled confounder is listed in the study text, "We also do not yet have data on distance to nearby city." Matching was based on case rates in June/July 2020 and wouldn't ensure equal case rates in Nov-March, and wouldn't address changes in seropositivity driven by work migration anyway. The number of seasonal urban workers returning to or leaving villages could have driven differences in seropositivity by adding / depleting donors who got infected in Dhaka / Chittagong (where previously-reported seropositivity rates were very high). This isn't even necessarily distance-based.
I think Jason did a good job of defending his position. My takeaway was that there was some, but little benefit to masking, but it was hard to tell the degree because the controls were not as aggressive as desirable. Sounds like more studies are needed. That said, his call for the less knowledgeable to accept the consensus belied his remark that he did not take either side in the argument and simply sought the truth. I suspect he was biased going in and there is a good likelihood that skewed his work in interpreting some of the study areas with weaker controls.
This interview was challenging to listen to. There was so much interrupting and talking over that it wasted time and took away from the value of the debate.
I listened to it while driving, otherwise would have given up. I need to go to the actual study now to see what was being debated. I did learn a lot though. As always thank you James.
My impression.
This showed how easy ti can be to misrepresent a study. So many factors and considerations go into building a viable study. Dr. Abaluck's study was a huge under taking and is to be commended for it. But his defensive posture made it difficult to listen to even if he was making a valid point. This suggests he had some kind of preconceived notion about how it would go down. Had he let his guard down he would have made his points more effectively to layman's like me because he's clearly competent. Still, he deserves credit coming on to discuss it. We need more of these debates.
That all being said. I'm not convinced masks work.
When I was a stock broker I'd hear all sorts of reasons and explanations form a company justifying their quarterly results. Or read about why a company posted a result but we needed to read the details. For sure, that was at times necessary but at the end of the day I asked a simple question, despite all the qualifications, do you make money? Are you profitable? Yes? No? Science has p-value, investments P/E multiple. And like Abaluck expressed not being a fan of p-value, there were investors who took P/E with a grain of salt or at least cautioned interpreting data based on it alone.
I apply the same thing here with the masks. I note that it's the rule and not the exception that data shows masks have marginal benefits. Yet, we always have to hear about caveats. Bottom line is do they work or not?
As for my impression of the types of masks, cloth masks are akin to t-shirts around our mouths. Useless. So enough of this cloth nonsense. As for surgical masks, we've had decades worth of studies - in both hospital and community settings - that showed they're ineffective. The talk about N-95 is also somewhat misplaced because they're simply not feasible in community settings. I wore those in construction and you'd better change them every coupe of hours....
I looked up Dr. Abaluck and see he is a professor of economics. Economics is the art of obfuscation and getting any data to say what you want or exclude it as an externality or some other construct useful for deception. It is not a science just as political science is not a science. I find it preposterous that anyone could possibly think they can barge into a third world country and get people to conduct some kind of behavior for the purposes of your study and then measure accurately the result of that behavior especially for something as nebulous as covid prevalence measuring. I really wonder how many people wore those masks as they were trained to do (surely they were trained just like medical professionals get trained in ppe usage). Did they make sure they fit well? Remember all it takes is openings greater than 3.2% of total mask area and a mask filters nothing. (https://www.tandfonline.com/doi/full/10.1080/02786826.2020.1817846) His vaunted surgical mask never fits well with openings all around the mask, just look at people wearing them(see the interviews with industrial hygienists). So we have professionals (Industrial Hygienists) whose job is the protection of people from airborne hazard and has a mature and stable base of principles and practices for achieving this. They say cloth masks don't work for viral particles, they say surgical masks don't work either. Now we have an economist who gets a bunch of people in Bangladesh to wear masks around and measures covid rates and sees if he can find a correlation. I have an even better one to study which is analogous to masking. The State of Utah has one of the lowest rates of Covid deaths per million in the country. Most people in Utah are Mormons, Mormons wear a particular garment as a part of practicing their religion. They believe that garment protects them and I am not here to say they are wrong or demean their religion. A mask is also a garment that people wear because they think it protects them even though the people whose job it is to figure out how to protect people from breathing in bad things say they don't. So, if some other economist out there decided that he wanted to prove the Mormon garment absolutely does protect from Covid I'm sure he could; just as Dr. Abaluck went to Bangladesh and got a bunch of people to wear masks and purported to be able to glean some sketchy data saying it protected them from Covid. Bottom line, if you want some tobacco science done get an economist to do it, they are so good at getting any data to fit any theory they might come up with. A better debate would have been between Industrial Hygienist Steve Petty and the co-authors on the study who Dr Abaluck referred to as "knowing more than you about masks". Then we could get some meat into the debate, does a given mask filter out virus? Can you get people to wear them properly for long periods of time given how uncomfortable it is?
Of course you can do simplistic mechanical studies (cited by Dr. Abaluck of coughing into a petri dish masked and unmasked) that don't really study how a mask is used but that is what economists are good at citing as externalities the fact that users don't follow their strict models and so are excluded. The fact is most people wear masks as clothing and are just complying and are not going to go to the trouble of wearing these things 'properly'. They are constantly messing around with them because they are uncomfortable and so their hands are constantly contaminated as is everything they touch. This is especially true of children.
When it comes right down to it, it is everyone's right to breathe freely without obstruction. The process of respiration is not defecation. This idea of required face garment is extreme and brutal and those who push it are despots.
My takeaways are pretty straightforward. I have very little expertise with statistics or designing/executing/evaluating scientific studies. That disclaimer aside, it seems to me that:
1) Your point regarding the language used to characterize the results of the study in the media are well-founded. The headlines you chose were actually the somewhat less bombastic and triumphant examples that were paraded about. Words have meaning and too often have the impact of distorting perception—the gymnastics Abaluck went through in qualifying his support of the headlines you cited were very revealing.
2) The primary findings of the study seem relatively weak, especially in light of ‘symptoms’ and the huge overlap between Covid-19 and other relatively common conditions. The applicability of this study’s findings to the Covid pandemic—specifically—doesn’t seem strongly supported, especially as it relates to the question of whether or not masks reduce infection or transmission of SARS-CoV2.
3) The social aspects, especially cultural factors, seem understudied. The specifics of Bangladeshi culture, norms, behaviors, customs, etc., are only vaguely understood. Given that masking is such a socially-entwined intervention, it begs the question of extent of applicability of this study to other cultures (the US, for example). This isn’t a question that is adequately addressed, but would seem to be fairly pivotal.
Though it was a long debate, I found it very worthwhile to watch in its entirety. You, Dr. Jack, did a great job of staying cool as a cucumber and patiently (persistently) teasing out the dialogue despite his initial defensive posture. I think there was a kind of (rickety) bridge built here, which is encouraging. Thank you!
None of the masks studied are for PATHOGEN PROTECTION. I think that is all we have to say here IMO. I would ask Dr. Abaluck: were any of the masks used in the study actually graded for pathogens? The answer is simply: NO. The rest is blah blah blah. The petri dish??? Isn't that bacteria???? He mentions at the beginning the cost-benefit ratio of masking. Now, I didn't read the study, but intend to. My question is, did they actually take into account reduced immune function etc etc from masking? Was it true cost-benefit analysis? If so, what affect did this have on findings? My guess is that they didn't and it does. So again, researching for just the spread of coV-2 and not reduced health outcomes....hmmm....reminds me of another debate.
Searching out this study showed me the clear bias in the science world as compared to real critical thinking. If this study only showed a 1% decrease in symptomatic cases between cohort, I find that result a nothing burger. How are the other percentages calculated....another question for Dr. Jack.
"The proportion of individuals with COVID-like symptoms was 7.62% (N=13,273) in the
intervention arm and 8.62% (N=13,893) in the control arm. Blood samples were collected
from N=10,952 consenting, symptomatic individuals"
So 10,952 were tested......from?? and of 300,000 individuals for 5 months, only 10,952 tested positive?? Were there other treatments administered? (HCQ, Ivermectin etc). Also, how many were already immune? I have more questions than answers. Another bogus study.
https://www.poverty-action.org/sites/default/files/publications/Mask_RCT____Symptomatic_Seropositivity_083121.pdf
The only way they'll ever really be able to tell if any kind of mask has any effect on stopping transmission is to do human challenge studies where different groups of people wearing different types of masks for different lengths of time are purposely exposed to a respiratory virus (as I explain in this blog post: https://beyondspin.wordpress.com/2022/02/08/to-mask-or-not-to-mask-that-really-isnt-the-question-or-necessarily-the-solution ) . For those types of studies, you have to set aside ethical constraints....which obviously is very problematic. Otherwise all these studies, that wait for people to get infected, will always have a myriad of confounders no matter how much the researchers believe that they'e controlling for various variables. The aRR on the particular study you two are debating about isn't the slightest bit compelling, and other meta-analysis of mask studies showing rRR's of much higher rates (53%) have very weak confounded aRR's of around 0.5%. So again, not the least bit compelling. So even these RCT's are pretty much bull shiet.
#
If I'm getting this right, Dr. Abaluck is saying that the likelihood of any confounder in the study occurring is so low that it is pointless to consider it. Such as where another factor, like last names ends up correlating with the positive mask outcomes nearly perfectly. What you're saying is that a confounder occurring ( whatever it may be ) is far more likely than is usually assumed.
So there could be something else along with mask wearing that lead to the appearance of a statistically significant result for surgical masks. Last name ( genetics ), change in behavior of either the wearer, or the people they interact with, etc. Abaluck is saying there is virtually no chance this happened with enough overlap to the results to make it worthwhile to consider. Would that be a fair summary?
I personally find it very hard to believe that this study had the proper controls in place to make any result meaningful. There are far too many other factors involved. You would need to have both groups be so similar in nearly every aspect while at the same time conducting the study in the real world as to make the ability to do the study itself basically impossible.
So I assume that Dr. Abaluck did not get the $1 million.
IMHO he's too defensive and overly animated with some responses which kinda shows maybe he doesn't even believe what's in the study. Too emotional.
Thanks so much for making this information available!