RESEARCH ARTICLE


Assessment of External Quality Assurance Scheme Participation Level, on Salmonella and Shigella Species



Firehiwot A. Derra*, Bisrat H. Mariam, Tekilil Biza, Tesfaye Legesse, Redwan Muzeyin, Samson Girma, Yosef Beyene, Almaz Gonfa, Gonfa Ayana, Eshetu Lemma
Department of Food Science and Nutrition Research, Ethiopian Public Health Institute, Addis Ababa, Ethiopia


Article Metrics

CrossRef Citations:
0
Total Statistics:

Full-Text HTML Views: 387
Abstract HTML Views: 295
PDF Downloads: 164
ePub Downloads: 129
Total Views/Downloads: 975
Unique Statistics:

Full-Text HTML Views: 248
Abstract HTML Views: 173
PDF Downloads: 111
ePub Downloads: 87
Total Views/Downloads: 619



© 2019 Derra et al.

open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: (https://creativecommons.org/licenses/by/4.0/legalcode). This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

* Address correspondence to this author at the Department Researcher at Public Health Microbiology Laboratory Gulele Sub-City, Patriot Street, P.O.Box 1242. AA, Ethiopia; Tel: +25 1941246677; E-mail: firehiwot_a@yahoo.com


Abstract

Background:

External Quality Assurance Scheme (EQAS) is the system which allows every laboratory to compare its overall performance with other internal and external existing laboratories, working in similar disciplines. Significant improvements were reported in different laboratories and countries after attending one or more of such programs. The project objective was to assess EQAS participation level in Salmonella and Shigella species that had been processed for six years under WHO-AFRO GSS EQAS program.

Methodology:

Samples received for Salmonella and Shigella species, as well as Campylobacter and other unknown enteric pathogens identification were directly inoculated to the suitable and selective media according to the type of organisms. Serogroups were reported using terms according to Kauffmann-White-Le Minor procedures. For antimicrobial susceptibility testing, drug diffusion method and CLSI interpretation guideline was used.

Results:

From the overall participation (2008-2013), serogrouping results were correctly reported as 62/ 71 (87%). None of the deviations was recorded for Shigella species. Participation for Campylobacter species was only twice per six years, in 2009 and 2010; the results of agreement with the expected values were ½ (50%) and 2/2 (100%) respectively. In line with this, the antimicrobial susceptibility participation was correctly reported as 320/356 (89.9%).

Conclusion:

Even though everyone has gained knowledge and awareness about the benefits of EQAS by default, its acceptance and implementation in developing countries are less communicated and exercised. The final recommendation will be that all higher officials and policymakers in the field have to give attention to it and allocate adequate budget on a continuous basis.

Keywords: Serogrouping, Serotyping, Antimicrobial susceptibility testing, Campylobacter, Enteric pathogens, (EQAS).



1. BACKGROUND

EQAS (External Quality Assurance scheme) is a system which supports every laboratory to provide effective service to the consumers and maintain their performance quality based on the standard requirement. Two types of quality assurance approaches are well known; the intra laboratory comparison and intercomparison, which are characterized by the assessment of the laboratory by other laboratories working in the country and a periodic laboratory auditing with cooperating laboratories working in different countries, respectively. The latter is defined as EQA or proficiency testing and it is a mechanism of proving the overall laboratory performance towards the expected standard of the specific discipline [1-3].

According to the World Health Organization guideline draft 2002 report, the quality of service has been demonstrated to improve after participating in different EQAS programs. In agreement with this, the South African hospital accreditation program proved that statistically significant improvement was achieved on a number of quality indicators as a result of continuous participation [4].

With this regard, there are some national laboratories that provide EQAS participation opportunities for some specific programs [5-7]. An Iranian national external quality assessment scheme can be taken as a model by providing quality testing materials for both governmental and private microbiology laboratories found in the country [8, 9]. This active program was launched in 1994 and covered a wide range of clinical microbiology parameters performing two or three runs annually to evaluate their overall performance through periodical time frame [10, 11]

The Tokyo Metropolitan government, on the other hand, had given a license for independent microbiology laboratories working in Japan and have been supported to involve in such mandatory evaluation since 1982. In the year 2000, they have assessed their eighteenth years' performance and estimated the impact of the program to identify the success and limitation [8, 12], similar to French, Ontario, and Indian national laboratories [13-15]. In Edmonton, Alberta, Canada, the detection skill of extended spectrum lactamase producing organisms which was 10% in 1997 was amplified to 100% by the year 2008, after serial participation. Laboratories also have revealed progress in reporting and interpretation of third-generation cephalosporin test results [16]. In comparison to Canada, the UK, NEQAS scheme for checking the CDC recommendation (use of liquid culture techniques to improve the time taken to diagnose TB infection within 21 days), the percentage of participants reporting positive results had been ascending from 55% in 1995 to 83% in 2002 [17].

From a global aspect also, there are different organizations which have EQAS/ PT programs and provide samples for laboratories found worldwide. CLSI, ISO, CDC, and WHO are among other organizations. The WHO-AFRO Global Salmonella-Surveillance EQAS program known as a global network of epidemiologists and microbiologists is involved in Salmonella surveillance. The first one initiated in the year 2000, and since then continuously serving all the developed and developing countries by providing quality control samples for free and closely supporting to upgrade their performance [4, 18]. In Africa, the National Institute for Communicable Diseases of South Africa is running a national policy of EQAS with well-functioning accreditation system and is also actively engaged in this program. On top of that, it provides external quality assessment programs on meningitis, plague, tuberculosis, mycology, enteric bacteria, malaria and on other very important disciplines. Despite the fact that the level of participation differs for most of the African countries, Ethiopia is currently more focusing on malaria, HIV, TB, polio, and in few bacteriological tests [19-22].

The public health microbiology laboratory is one of these laboratories working under the Ethiopian Public Health Institute and has only been participating in WHO-AFRO EQAS program of Salmonella and Shigella species, serogrouping and antimicrobial testing since the year 2008. This program encompasses four different testing schemes; Serotyping and/ or serogrouping as well as antimicrobial susceptibility testing of Salmonella, Shigella, Campylobacter species and identification of an unknown enteric bacterium [23, 24]. Nevertheless, the laboratory has been making an effort to widen its scope and participate in all schemes of the program, but the participation level is restricted only to few parameters.

Therefore, this evaluative proposal targets to evaluate the laboratory performance towards this specific program and to clearly show the current limitation that restrains it from active participation to this and other related quality assurance programs, similar to other developing countries [25, 26].

2. METHODOLOGY

This study is an evaluative retrospective study on EQAS participation level of Salmonella and Shigella species and covers six years period from the year 2008 to 2013. It was carried out at public health microbiology of EPHI. The study incorporated all EQAS results which had been processed within the indicated period of time. The Salmonella and Shigella strains and the ‘unknown' bacterial isolates were shipped as agar stab cultures and lyophilized for some specific reference strains like Campylobacter. They categorized as UN3373 biological substance category and transported through cargo transportations system annually.

On arrival of the samples to the laboratory, the agar stab culture, subcultured on suitable broth, and the stocks were organized for storage in a -80°C freezer. For reconstitution of the Campylobacter reference strains, there is an instruction for opening and reviving lyophilized cultures as attached inside the package.

The broth suspension then directly inoculated to the suitable and selective media as stated by WHO-EQAS procedure for each type of organisms. XLD or Mackonkey media were used to isolate among Salmonella, Shigella, E.coli and unknown samples at the first line, followed by using basic media like nutrient agar and blood agar for purifying the isolates before testing for serogrouping or serotyping. The Salmonella and Shigella strains were serogrouped/serotyped using the method routinely used in the laboratory. Sero-groups were reported using terms according to Kauffmann-White-Le Minor procedures [1].

In line with the WHO-EQAS protocol, all Salmonella and Shigella strains, as well as the E. coli ATCC 25922 reference strains were tested for susceptibility testing for the following antimicrobials:Ampicillin, AMP; cefotaxime, CTX; ceftazidime, CAZ; ceftriaxone, CRO; chloramphenicol, CHL; ciprofloxacin, CIP; gentamicin, GEN; nalidixic acid, NAL; streptomycin, STR; sulfamethoxazole, SMX; tetracycline, TET; trimethoprim, TMP and trimethoprim + sulphonamides, SXT. Although the additional testing of Extended-Spectrum Beta-Lactamase (ESBL) producing strains, using the antimicrobials CTX and CAZ in combination with the inhibitor clavulanic acid is another opportunity, the laboratory was unable to participate. The following antimicrobials were used for AST of Campylobacter strains: chloramphenicol, CHL; ciprofloxacin, CIP; Erythromycin, ERY; Gentamicin, GEN; Nalidixic acid, NAL; streptomycin, STR; and Tetracycline, TET.

Table 1. EQAS serogrouping result of enteric pathogens, 2008-2013 (all the data were copied from WHO-Afro EQAS annual reports from 2008-2013).
Year of Participation TN of Isolates Serogrouped % of Correctly Serogrouped Isolates (P) % of Outlier (probability of an Error (1-P) Standard Error 95% CI
2008 8 7 (87.5%) E1 (12.5%) 0.117 0.65-1.104
2009 12 11 (91.7%) D2 for D1 (8.3%) 0.079 0.76-1.07
C. jejuni (2) C.jejuni (1) (50%) C.coli for C.j
(0.5)
V.mimicus (1) Vibrio (1) -
2010 12 11 (91.7%) (8.3%)
G (O:13)Not reported
0.079 0.76-1.07
C.jejuni (1) C.jejuni (1) 100% -
C. coli(1) C. coli (1)100%
S.enterica (D1&D2) (1) S.enterica for Citrobacter spp
2011 8 8 (100%) - -
2012 12 11 (91.7%) E4 for E1 (8.3%) 0.079 0.76-1.07
S.paratyphi (1) - S p for S.arizona
2013 12 11 (91.7%) A for C2-C3 (0.083) 0.079 0.76-1.07
Total 71 62 (87%)
Table 2. Deviations on susceptibility pattern of tested isolates per expected result (all the data were copied from WHO-Afro EQAS annual reports of 2008-2013).
Years of Participation No. of Antimic-robial Agents Tested No. of Antimicrobial Agents Tested Correctly Tested Incorrectly Standard
Error (SE)
95%CI Minor Deviations (S to I or I to R switch) in Number Major Deviations (S to R switch)
In number
Very Major deviations (R to S switch)
In number
Critical Deviations
(R to S /S to R switch)
(Major + Very major)
%
Total Deviations
2008 80 73 (91.7%) 7 (8.75%) 0.032 0.86-0.98 5 - 2 2 7 (8.75%)
2009 108 99 (91.6%) 9 (8.3%) 0.027 0.86-0.97 4 - 4 4 8 (7.4%)
2010 84 76 (90.4%) 8(9.52%) 0.032 0.84-0.97 1 - 7 7 8 (9.5%)
2011 -
2012 84 72 (85.7%) 12 (14.3%) 0.038 0.78-0.93 5 1 3 4 9 (10.7%)
2013 -
Total 356 320 (89.9%) 15 1 16 17 32 (8.9%)

Based on the single arm statistical formula, performance deviation and agreement with the reference expected results were calculated using a single proportion method in different categories. The percentage of obtaining satisfactory/unsatisfactory results was also calculated as a summarized report. SPSS version 20 was used for data entry by transferring from the annual report. For result interpretation and analysis, the standard error of the true proportion P value was calculated to observe the level of statistical variation at 95% CI. This research was ethically cleared by EPHI Scientific and Research Ethical Clearance Committee (SERO) directorate.

3. RESULTS

Among all serogrouped isolates, 62/ 71 (87%.) were correctly reported Table 1. Likewise, 320/356 (89.9%) antibiotics were verified in agreement with the actual results for antimicrobial susceptibility testing. The highest major disagreement was observed on ciprofloxacin (6X) and tetracycline (5X). On the contrary, the minor deviation was observed on tetracycline (5X), followed by Nalidixic acid (3X), in decreasing order Table 2. Majority of the antimicrobial susceptibility testing result discrepancy was observed with Salmonella Subgroup C, for 8 to 10 different types of drugs. Alternatively, the highest deviation was seen by Tetracycline, with subgroups B, C, E and F. In contrast, none of the variations were observed against Salmonella Group A, G and H Table 3.

4. DISCUSSION

It is believed that quality assurance is an important element in the provision of quality public health services, and to improve the performance of health laboratories. In spite of continuous endeavours on the implementation of EQAS in the developing countries, even for more than a decade, it is not as successful as it should be. Despite the fact that EQAS participation is mandatory to upgrade the overall performance of the laboratory, most of the developing countries’ involvement is far from what is the expected. Even internal quality control activities have not been accomplished regularly in many laboratories [32].

Achieving the quality issues at continues basis is becoming more difficult for developing countries. These broad limitations have been observed in all African countries except South Africa involved in all programs, since the year 2001. According to the WHO-EQAS 2012 report Fig. (2) [19], Tunisia, Algeria and Kenya are the best participants from Africa following South Africa, when compared to other African countries. In contrast, most other countries have not shown any participation. A very similar limitation was also reported in South East Asia regions. Some of the countries in the region have been participating for all or some of the parameters. China is a well-known country which has been participating in all schemes reminiscent to North and South American countries. The Oceania region members, Australia and New Zealand are also the major participants resembling European countries. Ethiopia has been continuously participating since the year 2008 but not fully as expected. This restrains further improvements in the laboratory .

Although there are some laboratories participating in different specific programs, inconsistency in participation level has been observed as a result of different factors. One of these factors is the budget limitation. Likewise, the public health microbiology laboratory (Ethiopia) was enrolled in this program after the year 2007 following WHO-AFRO GFN/ GSS II training. Participation had been continuous beginning from 2008 to 2013, though the level of involvement varied each year and only targeted towards a single program Fig. (1). Nevertheless, the participation was only limited to serogrouping and sometimes for antimicrobial susceptibility testing, the laboratory could not go further to serotyping. The reason was either the kits were not available according to the program criteria or were incomplete. In agreement to this, antimicrobial testing participation was missed twice, in the year 2011 and 2013.

Comparable results have been obtained from different countries. The WHO-AFRO GSS program review report of 2000-2007 [18] explains that each year, the overall per-formance of Salmonella serogrouping for all participating countries is satisfactory as the percentage of deviations is very low for all of the test strains. The public health microbiology laboratory (Ethiopia) performed acceptably in this respect. The deviation in each participation year was less than 10% except, for the first year of participation (2008), which was 12.5% [18]. None of the discrepancies were also recorded for Shigella species since the confirmation procedure is more accurate and very few weak reactions have been observed. The possible reason might be that few numbers of known species are circulating in the environment, unlike the different complex nature of Salmonella species. In general, the overall variation in serogrouping is statistically insignificant since the calculated P-value is > 0.05.

Participation for Campylobacter species was done only twice per six years, only in 2009 and 2010; the results of agreement with the expected values were ½ (50%) and 2/2 (100%) respectively. The challenges were mostly observed in the identification of Campylobacter species, because of difficulties in the optimization trial of hippurate hydrolysis test. The resulting interpretation of biochemical testing which has a different range of acceptance for positive/negative categories (like > 90% or >99% or >50%...) and the emerging nature of new strains which can never be ruled by the existing previous rules also play a role in the limitation of identification of organisms, using conventional methods. The interpretation of Cephalosporin and Nalidixic acid susceptibility testing, the two drugs which are used for identification purpose of Campylobacter, sometimes confronts challenges since the guideline is mostly applied for MIC methods (EUCAST) and disk diffusion technique and not much applied for this particular type of organism [27]. However, laboratories in most of the developing countries follow the latter method therefore, correlation of the results to MIC might create some discrepancies.

Table 3. Sub-groups of Salmonella/ Shigella species and their corresponding antimicrobial testing result deviations observed (per the accepted ranges) (all the information were obtained from WHO-Afro EQAS annual reports of 2008-2013).
Sub-Groups of
Salm/ Shig
isolates
Ampcillin ceftaxime Cefrazidime Ceftriaxone Chloamphenicol Ciprofloxacin Gentamicin Nalidixic
Acid
TMP/SMX Tetracycline total
G- A
G-B 1 2 3
G-C 1 1 2 1 1 4 2 1 13
G- D 1 1
G- E 2 2
G- F 1 1
G- G
G- H
Shigella species 2 2
Overall 22
Fig. (1). Six years trends of Salmonella and Shigella serogrouping and antimicrobial susceptibility testing participation.

Fig. (2). Countries participated in the WHO EQAS 2012 program are shown in green color, including Ethiopia. The white color indicates the non-participating regions for the indicated specific period. (Adopted from; the WHO EQAS 2012 Report, DTU Food).

In spite of the participation level for unknown samples being successful in most of the years, the results had not been as excellent as they should be, because of incomplete antisera or biochemical reagents.

Based on the WHO-AFRO goal to have all laboratories perform Salmonella antimicrobial susceptibility testing with a maximum of 10% total deviations (minor, major, or very major deviations) and a maximum of 5% critical deviations (major or very major deviations), the laboratory performance had met the requirement of total deviation in all participation years except the year 2012, which exceeds the limit by 4%. In parallel to antimicrobial susceptibility testing requirement of critical deviation, the criteria were made for three years; 2008 (2.5%), 2009 (3.7%) and 2012 (4.8%), except for the year 2010 which was (8.3%) [19]. Although the above results are acceptable as per WHO -EQAS criteria, the overall variation is statistically significant since the calculated P-Value is less than 0.05, which needs further attention.

The detection trend for serogrouping had shown a slight progress from 87.5% in the year 2008 to 91.7% in consecutive years up to the last year of participation that was 2013. In contrast, to serogrouping, successful participation to antimicrobial susceptibility had been decreased slightly from 91% of the previous years to 85.7% in the last participation year of 2012. It can be concluded that the overall participation trend has been well achieved even with slight differences .

This result, in general, indicates when the laboratory performance is well and satisfactory; it works for implementing standard protocols. The efficiency of laboratory professionals working on this program is also remarkable and competent even though slight outlier results were registered due to different factors. Similar results were obtained in different countries [28, 29]. For example, Indian EQAS system scheme also evaluated reference laboratories performance of North and South Indian regions on HIV- testing parameters and reported satisfactory results in general [30, 31].

CONCLUSION

Even though everyone has gained knowledge and awareness about the benefits of EQAS by default, its acceptance and implementation in developing countries are less communicated and exercised. The failure of incorporating and adopting such important programs to the laboratory system, as the WHO-AFRO's scheme expectations, ends up in building an underdeveloped (stagnant) laboratory, which works far from the modern and standard system. According to the observations from this report, most of the reasons to not actively engage in this program are associated with antisera and chemical scarcity in addition to some laboratory related issues . The main message from this observation is that all laboratories have to have a budget plan annually for EQAS program, similar to routine services, especially for the national reference laboratories. They have to mandate routine EQAS program participation that meets an international quality standard [32]. All necessary chemical and materials have to be supplied to these laboratories on a consistent basis. Therefore, it can be taken as a part of the usual normal activity of the laboratory. Unless such mechanisms have been developed and implemented, the program’s usefulness will not be traced out.

It can be concluded that the overall drawback is not associated with budget scarcity in the area, it is an overall awareness limitation. Most of the responsible agents like WHO have to continue supporting the implementation strategy of EQAS participation. They have to follow the laboratory performance, donate kits and minimize the occurrence of errors. Nowadays, the gaps between developed and developing countries have increased not only in terms of participation in such programs but also in the research scope, which have made the latter more dependent on the former. Therefore, minimizing the globalization issues and the gaps between the two groups have to be the main agenda to be implemented by the active players worldwide.

Moreover, if the laboratory performs year after year based on its own in-house method, it can never evaluate its position against the international standard. Whenever there are quality assurance issues, EQAS is mandatory at all stages. Furthermore, the links between the regional and the central laboratory have to be strengthened so that the participation scheme will be extended , up to the peripheral level.

Therefore, the final recommendation for the improvement of this program is all higher officials and policymakers in the field have to pay serious attention to this issue [33, 34]. Ethiopian and other African laboratories working in similar disciplines have to be equivalently competent in doing scientific research and modernizing their laboratory activities, in a similar way to that of key member countries. Providing awareness of this discipline is the major and main goal of this review report and the WHO- AFRO GSS program scheme [22, 33, 35].

LIST OF ABBREVIATIONS

AIDS = Acquired Immunodeficiency Syndrome
AMP = Ampicillin
ATCC = American Type Culture Collection
CAZ = Ceftazidime
CDC = Center for Disease Control and Prevention
CHL = Chloramphenicol
CIP = Ciprofloxacin
CLSI = Clinical Laboratory Standards Institute
CRO = Ceftriaxone
CTX = Cefotaxime
DFID = Department for International Development
DTU = Danish Technical University-National Food Institute
ECDC = European Centre for Disease Prevention and Control
EPHI = Ethiopian Public Health Institute
EQALM = External Quality Assurance program in laboratory medicine
EQAS = External Quality Assurance System
ERY = Erythromycin
ESBL = Extended-Spectrum Beta-Lactamase
EUCAST = European Committee on Antimicrobial Susceptibility Testing
EURL AMR = European Union Reference Laboratory for Antimicrobial Resistance
GEN = Gentamicin
GFN = Global Foodborne Infections Network
GSS = Global Salmonella-Surveillance
HIV = Human Immunodeficiency virus
ISO = International Organization for Standardization
MIC = Minimum Inhibitory Concentration
NAL = Nalidixic acid
PT = Professional Testing
SMX = Sulfamethoxazole
STR = Streptomycin
SXT = Trimethoprim + Sulphonamides
TB = Tuberculosis
TET = Tetracycline
TMP = Trimethoprim
UK NEQAS = United Kingdom National External Quality Assessment Service
WHO = World Health Organization
XLD = Xylose Lysine Deoxycholate Agar

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

Not applicable.

HUMAN AND ANIMAL RIGHTS

No animals/humans were used in the study that is the basis of this research.

CONSENT FOR PUBLICATION

Not applicable.

CONFLICT OF INTEREST

The authors declare no conflict of interest, financial or otherwise.

ACKNOWLEDGEMENTS

The authors would like to acknowledge Ethiopian Public Health Institute, for allowing to conduct this study . The Food Science and Nutrition Directorate and the Public Health Microbiology section are also acknowledged for creating such a suitable environment for this study to meet its main goal. We would also like to thank WHO AFRO GFN/ GSS network free participation program, especially for Dr Rene Hendrickson and Mrs Suzanne Karlsmose for giving such a remarkable opportunity for microbiological laboratories and giving a chance to test their performance in a continuous yearly basis.

REFERENCES

[1] Patrick ADG. Antigenic formulae of the Salmonella serovars in WHO Collaborating Centre for Reference and Research on Salmonella. Kauffmann-White-Le Minor 9th. 2007; 45-64.
[2] Montagu D. Accreditation and other external Quality Assessment systems for health care- DFID 2003; 1-40.
[3] Banoo S, Bell D, Bossuyt P, et al. Evaluation of diagnostic tests for infectious diseases: general principles. Nat Rev Microbiol 2006; 4(9)(Suppl.): S21-31.
[4] Cavaco LM, Aarestrup FM. External quality assurance system. J Clin Microbiol 2009; 47(9): 2751-8.
[5] Chaudhary R, Das SS, Ojha S, Khetan D, Sonker A. The external quality assessment scheme: Five years experience as a participating laboratory. Asian J Transfus Sci 2010; 4(1): 28-30.
[6] Rathod K, Patel K, Gupta M. Five years’ experience as a participating laboratory in External quality assessment scheme-NABH accredited blood bank from the rural area. Int J Adv Res (Indore) 2014; 2(7): 1248-51.
[7] Kristensen GBB, Nerhus K, Thue G, Sandberg S. Results and feasibility of an external quality assessment scheme for self-monitoring of blood glucose. Clin Chem 2006; 52(7): 1311-7.
[8] Marandi FR, Rahbar M, Sabourian R, Saremi M. Evaluation of Iranian microbiology laboratories for identification of etiologic agents of bacterial meningitidis. Survey results of an external quality assessment scheme (EQAS) programme. J Pak Med Assoc 2010; 60(1): 48-51.
[9] Abbassi M, Rahbar M, Hekmat Yazdi S, Rashed Marandi F, Sabourian R, Saremi M. Evaluation of the 10th external quality assessment scheme results in clinical microbiology laboratories in Tehran and districts. East Mediterr Health J 2006; 12(3-4): 310-5.
[10] Hastings RJ, Maher EJ. Quell horst-Pawley B, Howell RT. An internet-based external quality assessment in cytogenetic that audits a laboratory’s analytical and interpretative performance. Eur J Hum Genet 2008; 21(16): 1217-24.
[11] Raahbar M, Sabourian R, Yazdi M, Roodokai M. Evaluation results of 21st iranian external quality assessment schemes (EQAS) of microbiology laboratories in 2007. The internet J of Inf Dis 2008; 7(1): 1-6.
[12] Kumasaka K, Kawano K, Yamaguchi K, et al. A study of quality assessment in clinical microbiology performance of independent laboratories in Tokyo: 18-year participation in the Tokyo Metropolitan Government External Quality Assessment Program. J Infect Chemother 2001; 7(2): 102-9.
[13] Jesudason MV, Mukundan U, John TJ. External quality assessment scheme in microbiology in India. Indian J Med Microbiol 1995; 13(1): 26-9.
[14] Ontario Medical Association (OMA). Quality Management Program-Laboratory Service. General program information 2014; version 5.3
[15] Guilliard O, Piriou A. Pineau (1996) French external quality assessment schemes for a lead in blood and aluminum in plasma and dialysis water. Ann. 1st. Super. Sanita 1996; 32(2): 241-5.
[16] Restelli V, Barteluk R, Kwok E, Rennie R. Quality assessment. in, european organization for external quality assurance program in laboratory medicine (EQALM). Progress and improvement for identification of extended-spectrum beta-lactamases (ESBLs) through external quality assessment. An Int J of Qual Assura 2010; 20(1): 1-37.
[17] James V. European Organization for External Quality Assurance program in laboratory medicine (EQALM).. Long-term evaluation of Qualitative EQA results, UN NEQAS for Microbiology. An Int J of Qual Assura 2010; 20(10): 1-37.
[18] Hendriksen RS, Seyfarth AM, Jensen AB, et al. Results of use of WHO Global Salm-Surv external quality assurance system for antimicrobial susceptibility testing of Salmonella isolates from 2000 to 2007. J Clin Microbiol 2009; 47(1): 79-85.
[19] Hendriksen RS, Karlsmose S, Jensen AB, Aarestrup FM. The External Quality Assurance A system of the WHO Global Foodborne Infections Network: Lyngby: National Food Institute, Technical University of Denmark, WHO- EQAS 2012 Report 2014.
[20] World Health Organization /National Institute for Communicable Diseases. External Quality Assessment Program Joint Annual Review Meeting Report, Johannesburg, South Africa 2013; 14th-15th, 1-17 2013.
[21] European Centers for Disease Control and Prevention. External quality assurance scheme for Neisseria meningitides Technical Report; Stockholm 2013 September; www.ecdc.europea.EU
[22] Petersen A, Aarestrup FM, Angulo FJ, Wong S, Stöhr K, Wegener HC. WHO global salm-surv external quality assurance system (EQAS): an important step toward improving the quality of Salmonella serotyping and antimicrobial susceptibility testing worldwide. Microb Drug Resist 2002; 8(4): 345-53.
[23] Karlsmose S, Cavaco LM, Hendriksen FS, Aarestrup FM. Comparison of proficiency testing results on antimicrobial susceptibility testing of Salmonella and Campylobacter obtained by laboratories from the ECDC FWD network (Public Health) and the EURL-AR network (Veterinary/ food) DTU Food, Lyngby 2013 August; 1-18.
[24] Clinical Laboratory Standard Institute. Performance Standards for Antimicrobial Disk Susceptibility Tests. Approved Standard 11thed Clinical and Laboratory Standards Institute The United States 2012; 32(1) MO2-A11
[25] Sapre JP, Chaudhari SN, Joshi HJ. The external Quality assessment scheme in hematology: Ten years of experience as a participating laboratory. Int J Med Sci Public Health 2014; 3(10): 1249-50.
[26] Kahlmeter G, Frown DF, Goldstein FW, et al. EUCAST (European Committee on Antimicrobial Susceptibility Testing). Technical notes on antimicrobial susceptibility testing 2006; 12(6): 501-3.
[27] Gutierrez G, Costa D, Espinet B, Carles Reverter J. The Spanish web-based external quality assessment scheme in cytogenetic of hematological malignancies: five years’ experience. National Congress Madrid. 2014; pp. 2014; 1-93.
[28] Community reference laboratory for salmonella Newsletter community reference laboratory for salmonella 2009; 15(1) ISSN 1572-3836
[29] Opartkiattiku N, Bejrachandra S, Byori R. Review on the external quality assessment schemes in Thailand. The Japanese J of clinical Pathol 2002; 50(2): 121-5.
[30] Sushi KM, Gopal T, Jacob SM, Arumugam G, Durairaj A. External Quality Assurance Scheme in a National Reference Laboratory for HIV Testing in South India. World J AIDS 2006; 2: 222-5.
[31] Sherwal BL, Gogoi S, Gupta P, et al. Achievement of quality standards in HIV testing of a state reference laboratory of north india through external quality assurance scheme: an evaluation of the 7-years’ experience. Int J Med Sci Public Health 2015; 4(6): 763-8.
[32] National Health Strategy 2011-2016.. National laboratory integration and standardization strategy 2013-2018. Qatar 2014; 1-56.
[33] Van Berkel HJ, Wolfhagen HA. The Dutch system of external quality assessment: description and experiences. Educ Health (Abingdon) 2002; 15(3): 335-45.
[34] Sisak F, Havlickova H, Hradecka H, Rychllik I, Kolackova I, Karpiiskova R. Antibiotic resistance of Salmonella spp. isolates from pigs in the Czech Republic. Veterinary Medicine 2006; 51(5): 303-10.: 0375-310.
[35] Yadav R, Bhartiya JP, Verma SK, Nandkeoliar MK. External quality assessment scheme (EQAS): Our experience as a participating laboratory. IJRRMS 2013; 3(4): 1-4.