[Cytometry] .fcs standardization/flow accreditation

Bill Eades eadesb at wustl.edu
Tue May 10 02:01:58 EDT 2011


Well, I started it......
In regard to accreditation, the sum of the efforts Mario itemizes is for naught when you factor my main point.  It is not whether or not new (or old) flow people need to learn this.  We all agree that we/they need it.  I feel that it was not a good choice for ISAC to get into this, because of its expense, and it being hard to sustain, especially when we are already being taught.  We have a pretty good system of teaching, given the publications, Flocyte, which uses hired guns respected by the flow community, the tenured flow experts themselves, within such Mario is always considered, and the vendors as well, no matter how exclusive their nomenclature gets, for their competition bears a "multi party system" which benefits all of us, most of the time.  Lets not forget this forum, with all of the best passive aggressiveness science can absorb along with the best topic related results you can get.  The resources exist, and are well promoted.  I see absolutely nothing gained by creating "card carrying" cytometrists but bureaucracy and our dues ill spent.  How much respect do you think I get from my 40 plus completion plaques from hardware, software and apps courses.  I put them in a closet, and they do not pack well for interview.  I am indeed railing against the accreditation concept in general.

Mario is absolutely right on the structure of ISAC, and I was wishing for what would be fair.  Interesting though, so much effort was made to show survey results last year, as if to justify, and sell us, on the already legislated decision.  I react to this most strongly, that our duly elected representatives were potentially deceived, as many are, by the presumption of fair sampling by survey.  I am pretty sure I am not alone, or really in the small minority, regarding this issue.  I am most likely stupid for not acting like most that just relent that "you cannot beat City Hall".  Just the same, it may explain why if implementing accreditation does not have the favorable reaction expected of the membership.  On another note, I will be sad for another decision if attendance in Baltimore wains, since it is now so necessary for us to have yearly meetings in an age where we are trying to "remote in" for as much as we can.

Software standardization.  I should have known there would be little empathy for this.  I am sorry that I may not have cited the exact keyword(s), but there has to be a way to key the scaling and transform type so that any offline analysis software can indeed plot as it was acquired and shown in the acquisition software.  This may not be necessary to comply with the requirements of the fcs revision, but it should be discussed further.  I do not think that software companies intend to be inaccurate or so disparate amongst their presentation, just that they bias to show off their adaptation of a transform as being "the one".  Does it not make you mad when you print plots in the acquisition software for sorting, and have to spent so much time forcing the third party software to scale similarly?  Think of having more than four third party softwares based on trying to accommodate users' patronage at a previous academic center.  Now you can call me stupid.

Clinical labs make their decisions on instrumentation to buy for their labs very much because of the way that data is reported.  I know from several clinical sources that they are not exactly happy with the way things are going right now in the research flow world, and cringe when they think of the history of research flow developing into clinical process.  If these same research disparities where forced upon clinical diagnostics my ranting rebellion would look puny, as it probably should.  The inability for analysis software to accurately and considerately present data by default is dangerous.  Those who say the matter is more complicated than this should seriously rethink it.  At very least, global preferences should be categorized by acquisition source, and be accessible easily by the user.  Seems like a trend is being set to "dumb down" flow, but no one wants to make it that much easier if you are not within the same set of vendors.  That is where the ethical dilemma starts.   

I am sorry but there "is such a thing" as inaccuracy of presented plotting, moreso now than ever.  No vendor on this forum could possibly be proud that both neophyte and accomplished reviewers of flow stammer over data presentations that were altered by a third party software because they did not know the secret handshake to make exceptions to the preferences.  Axis graduations are displaced, and populations become aberrant from typical distribution because of scaling errors.  Studies can take a wrong turn or become elongated beyond necessity, especially when compared to legacy plotting in pubs.  I have seen this, sometimes too late to do much about it.

You are only correct that the root plot coordinates are intact and that plots can always be fixed or the coordinates merely taken to spreadsheet.  The statistics are taken from the root coordinates as well.  Too bad we live in a short attention span world where the money plots are what we see on the publications, typically without the pre-exclusion.  No, I do not rely merely on pictures, but I know people and have met more that do!  I feel that Mario and I most likely share a silent disdain for the newbies coming up to us, asking for us to "make the data look like this!".

I am a little mad at vendors, for pointing fingers at each other, one less finger each to use for working towards a standard.  Software companies are blaming the instrument manufacturers for not readily giving them the keys to their kingdom, back at you for the widget maker to blame the software companies for intentionally excluding their product.  It ends up to be Vendors driving Science, instead of the reverse.  This is where we should be vocal, to at least let them know that they do not have to put out a survey for leading answers.

I cannot have the sole opinion of this forum that much more proficiency would be gained by our community by better standardizing flow results than by promoting accreditation to be implemented by a third party commercial entity.  When weighing the efforts necessary, the only thing in the way of standardization is getting the vendors together on the idea.  The rest is fairly easy and well within our technology.  

This is not meant to be an escalation, but I am responsible for what I have already put out there.  I respect Mario highly, and also expected a response from him, worth a whole pillar of salt.  ;o)  I do however have a defensible argument.  I would like to rest it here, if I am allowed, because I cannot be a slave to a forum.  It would be nice to have the time.  I also know my technical limitations, but feel that my thoughts on this deserved a chance, and would like for ISAC to merely consider them and not think this a diminishment of the association, because I really do not mean it as that.  I am not going to Baltimore because of protest.  I have to weigh travel, and "bang for the buck".  I am still not over the euphoria of Seattle, and my bag has not ripped yet.

I have already gotten the most response, privately, over my shortest, last part of my post.  Many of the vendors are asking who is spreading such information that scaling is the same as dynamic range, or, "is that what you mean?".  I apologize, I will not respond to this in writing, because, you know who you are.  This small subject is apparently all that Mario and I agree on.
Bill




Mario Roederer wrote:
> [Please note the multiple disclosures at the end.  I'm probably the single most-conflicted person on the planet with regard to this particular email.... so take my response with a few grains of salt.]
> 
> With regard to accreditation:  I guess it all depends on what is being accredited.  Which parts of the accreditation are you specifically against?  You presume to know the syllabus and topics already, but to my knowledge these haven't yet been defined.  
> 
> You rail against it in general, but I think everyone would agree that there are different ways to demonstrate facility with flow cytometry, and that those in charge of instruments could well show basic knowledge base in a way that has nothing to do with specific instrumentation.  For example, consider "understanding the basic theory and implementation of compensation."  Do you think that this would be unfair to demonstrate?  How about "understanding the roles and limitations of isotype controls"?  Or "demonstrating a basic knowledge of fluorochromes and reagents in order to assist with experimental design"?  Or, "How do you calculate the precision on a cell subset frequency to know significance?"  I'm pretty sure I can come up with a bunch of topics that are very general, of significant impact to flow cytometry, and are often misunderstood.
> 
> Note: there was no governance by survey.  Governance in ISAC is representational.  I.e., you voted for your ISAC councillors in order for them to make such decisions.  By voting for councillors, you entrust them to lead the society.  It is well established that you cannot govern a complex society by popular vote.  The survey was used primarily to gather feedback and guide the decisions of Council.  Anyone can run for Council, and I would strongly encourage this for those interested to do so!  And, of course, to provide Council with feedback -- which you can always do by private emails if you don't wish to do so on a public forum.
> 
> As Ryan noted, your criticism of the $PnE keyword is misplaced.  This keyword describes only the encoding of data in the FCS file; it need have no bearing on display transformations.  The latter was not encoded by FCS until the inclusion of the $PnD ("display") keyword a few years ago.  (Guess who championed that keyword.)  Even so, this keyword is only a suggestion, not a demand -- it is (rightfully) up to the software package to decide on the user interface for implementation.
> 
> I'm very concerned, though, about your comment: "...inaccuracy of the plots..."  There is no such thing.  Plots have no accuracy; they have no precision.  Only statistics (resulting from data aggregation and/or reduction) have accuracy and precision.  One should NEVER conclude anything quantitative from plots.  That's what statistics are for.  Plots should be used ONLY to visualize, guide, and provide some level of understanding of distributions.  But every plot type has its pitfalls; every plot can be mislead us in one way or another.  And certainly, this is doubly true for clinical use.  Please tell me you don't use graphical representations to make any sort of clinical decision.  That would be hard to certify.  Or accredit.
> 
> I do completely agree with you on the issue of dynamic range.  Every vendor is striving to "one-up" the next.  Claims of "7 logs of dynamic range" or whatever ... these are hogwash.  I don't care if the electronics IS capable of more than 5 logs of dynamic range (so please, vendor salespeople, do not email me claiming that your cytometer is better than others for this reason).  Biologically, 4-5 decades is the maximum useful range.  Most detectors have this as a useful range.  Just because your instrument goes to 11 doesn't make it better than one which goes to 10.  And... incidentally... let's remember that all the numbers on our axes, all the MFI values .... the magnitude of these numbers is entirely meaningless.  Doesn't matter.  No information.  (Please, don't email me telling me that your company's cytometer's values are relevant.  They're not.)
> 
> mr
> 
> 
> Disclosures:
> 
> (1) I sit on ISAC council, and voted for the accreditation program.
> (2) I was a co-author on the paper which first described automated algorithms for implementing display transformations (Biexponential / Logicle).
> (3) I have financial ties to FlowJo, which was the first to implement the display transformations, and does so in an automated way (which I still believe in).
> (4) I'm not a doctor, although I often play one in my mind.


On May 8, 2011, at 10:51 AM, Bill Eades wrote:

> Now that I have mustered enough courage to post about one thing, other related items about teaching flow have been heavy on my mind for a year now.  I am not going to Baltimore, so I probably need to say this here.  
> 
> Accreditation for flow cytometrists may on the surface seem like a good idea.  I realize that my minority opinion (assumed) is that this idea obfuscates what a potential employer would want, by merely having a person who may be good at tests be considered an expert.  What a person knows about flow and what they can do with it are two entirely different things, as I have found out as an employer.  ISAC is enacting an expensive endeavor involving third party entities to implement, adding yet another commercial factor to what we do.  In an expanding discipline such as ours, the predominant operator instruction is very vendor specific, and with so many new companies and offerings, a globalized method appraisal will only address commonality.  Also, different institutions and industries require differing responsibilities where flow cytometry is concerned, some involving sample preparation and biological experiment design, and some merely centralized operation of instruments and !
fi!
> le analysis.  With all of the branches of specialization, to try to cover the complexity with a fair and appropriate metric will be at best a token piece of paper.
> 
> While I am here, I will voice my complaint on governance by survey, by which this project was implemented by ISAC.  Lies, damn lies, and statistics.  If you are taking heart to a 20% sampling convention and dismissing the psychology of survey abstinence, you had better go back to popular voting to make ISAC decisions on projects this expensive and complicated, especially in this economy and funding climate.  My words will not matter, as I was told that the project is well under way, and as folks that did not participate in the survey are finding out, they lost their voice of governing thinking that survey may have meant survey and not vote.  My only hope is that this may be considered more closely in new decisions.
> 
> This may seem that my opinion is that vendors and possibly ISAC are bad, which is neither correct or justified.  It does mean that there is a much more dire project that is slipping away from us that needs attention now, being the enforcement of fcs keyword structure and standardization of plotting transforms.  Clinicians who have banked decades of pattern recognition have to scratch their heads, or just hand it to the tenured operators to figure out.  This is not to say that the evolution of 2nd order transforms is bad, because it has enhanced the ability to know how to interpret the evil dots on the axis lines with a value of 1.  What the problem is that we are having format wars in industry that bias towards vendor exclusivity and it is veering away from our need for standards. (gee, that never happened before...:o)) 
> 
> The largest problem is what I understand to be the nebulous and misunderstood $Pne keyword that is supposed to tell an analysis application how to scale an fcs file, and there might be others.  If you deal with the most popular instruments, and the most popular off line analysis suite, you are justified to say "what problem?", because they have worked this out.  If, however, you are experimenting with new designs from newer or less prominent manufacturing sources, often you are left in the desert, and you can in fact install inaccuracy of the plots.  The worst transgressor is the automated transform, which hides the preference to turn it off.  It is like having to check the box not to get spam when you shop online.  In the process of the analysis suite companies to show off their wares, the automated scaling and transform preference is at the least a distraction.  Consider a busy core with several analysis workstations that require complicated preference changes to merely !
lo!
> ok at an acquired file as it was acquired, and how in some s/w those preferences changes are global and carry over to the next user, who may not have the same file type. Management nightmare. 
> 
> The answer to this is to enforce the expansion of keyword specificity for the $Pne and other applicable keywords to cover all types of new transforms/digital gain factors, and to enforce software companies, existing and emerging, to by default display the acquired file as it was acquired by using the correct $Pne keyword, and not to bias towards unfamiliar view, whether considered better or not.  The promotion of novel transforms can still be there, but not by default, as this is so potentially dangerous, especially as this creeps towards clinical use.
> 
> Lastly, vendors, I implore you, please teach your salespeople that there is a huge difference between scaling and dynamic range. Thanks, and this most likely ends this installment of my bi yearly posting to the forum.  I apologize if this has been brought up in detail while my head was turned.
> Bill





More information about the Cytometry mailing list