Hi this is rob packard from medical device academy today i'm going to do a short video it's a live streaming video i'm in morning morgan grove uh in california i've on i'm on a trip i was doing a supplier on it and i have also been doing a mock fda inspection for a client but i think .

What i wanted to cover this week i wanted to stay on the topic of ozby is i'm out here doing a mock fda inspection and it also serves as an audit for that company an internal audit it also serves as a supplier audit because i'm auditing their supplier case their audit their supplier gets um audited by the fda are .

Inspected by the fda so a lot of people wonder well what is the difference between the different methods of auditing and which one should you use the fda uses accuse it manual to train all their inspectors and that tells you step by step how to do an fda inspection .

I don't know if you've read it you download it i'll put a link down below i'll add that after the video probably sometime this evening but the the q-sip manual is 108-page manual and that tells you how the fda does an inspection and they have four major categories they do .

Uh first the the kappa process second they'll typically look at design controls third they'll look at production process controls and fourth they'll do management systems and typically you'll have about a day spent on each of those areas and then the last day will be a closing meeting .

And they'll be writing their report and doing any follow-up on any records they weren't able to review before the inspection was done so that's what's involved in accusit inspection at a very high level a second approach is an audit checklist so most people that learn how to do auditing the first thing they're taught .

Is how to use an audit checklist how to create an audit checklist and all that is is you're going through whatever criteria you're using for your audit let's say it's iso 1345 or let's say it's the qsr 21cfr 820 and they're going to go into that and they're going to say oh i want to make sure they're doing this step .

So they'll put that on the audit checklist and they'll say you know is the is the company uh establishing supplier quality agreements according to 21 cfr 820.50 and and then is does the does that agreement include a phrase that says that the supplier .

Will notify the company of any changes to the product prior to implementing the changes so that would be an example something that would be an audit checklist and the third approach would be the process approach to auditing uh and there are other approaches here i've got a whole nother video just about .

The process of virtual auditing i'll provide a link on that but the the process approach to auditing the the first place you start is you usually start with a process interaction diagram for the company and a lot of companies don't have great process interaction diagrams but the purpose is to get an idea of the core processes .

From the beginning of you have an idea of a product for a customer or a customer user need and you carry that all the way through design purchasing production final inspection shipping and then .

Post market activities looking at complaints and gathering post-market feedback and then feeding back that back into the beginning of the process and when you do the process approach to auditing you don't usually have specific audits for some of the supporting processes like document control and .

Calibration the reason why you don't do that is because those processes can be sampled as you're doing the process approach to auditing so you have these three different approaches to conducting either an internal audit or a mock fda inspection which one should you use .

And one of the things that i would argue because a lot of people don't understand any of the three um in great detail they understand them at the very superficial level that i just described so they don't understand how similar the three should be they .

Just understand the differences at a very high level so they want to know which one should you use well one of the things that i see that's huge mistake that companies make when they are thinking about the q-zip manual .

Is the the fda they always come in and they ask for the procedure and then they ask for the log for that process so let's say i want to see your kappa procedure i want to see your capital log and then i'm going to sample capital records .

And that's essentially all they do when they're trying to do a mock fda inspection but there has to be so much more than that you have to understand how they manage the process you have to interview the person that's in charge of that process the fda does spend a lot of time on .

Records but you have to dig a little bit deeper and understand how the process works um the second approach that we were talking about the checklist approach when you have an internal audit checklist it's not about just asking do they have this yes no .

Don't ask the questions as a yes or no question when they teach that internal audit checklist approach what they're supposed to also be doing is teaching you number one always ask the question as an open-ended question so it's not yes or no it's um .

Can you please show me an example of your quality agreement with this how do you update and maintain your quality agreements with suppliers how do you how do you modify your quality agreement for software suppliers versus hardware suppliers how do you modify your .

Supplier agreement for off-the-shelf suppliers what do you do if a supplier like mcmaster carr won't sign an agreement those are the kinds of deep questions where you're not asking yes or no you're really digging in and when you fill in those checklists you're not just supposed to say yes they .

Did it was conforming you're not supposed to just write down the record what you're supposed to do is say what you looked at and what you look for and if you interviewed somebody which you also should have done you should be indicating who you interviewed and um .

What what that person gave you is information um in the in the interview and whether it was effective or not and indicate any audit trails you had so it's not just a yes or no they were conforming there should be some detailed notes about what you looked at .

And who you interviewed and how far you went and i would argue that you could use the process approach to auditing for both techniques and that's exactly what i do if i have a regulatory checklist so let's say it's an nd sap checklist and i'm looking for that supplier quality agreement that's a requirement .

Also over an mdsap and fda requirement everywhere if i'm looking for that supplier quality agreement i'm going to look at it with the context of the process approach to auditing i'm going to ask the seven questions that i was trained to ask as part of that turtle .

Diagram and there are seven steps number one can you please describe the process for establishing qualification of new supplier for example number two what are the inputs into the supplier qualification process so what documents do you request from various .

Parties to qualify a new supplier what are the outputs so this is step three what are the outputs of the supplier qualification process usually an asl approved supplier list a supplier quality agreement and an approval form indicating that you've approve
d the supplier and maybe even a schedule for .

When you're going to audit that supplier or re-evaluate that supplier or create supplier audit scorecards fourth you're going to be looking at what resources are required for the supplier qualification process so do they use any computers is there any uh process validation involved is there any calibration is there any maintenance .

Any equipment any space any facility that's required for that process because the supplier qualification process um is probably not going to be something that gets calibrated or requires maintenance you might think well i don't have to worry about this it doesn't require a .

Physical facility we just do it in do the supplier qualification process at our cubicle in our office or in a conference room as a team but the one thing that we do see a lot of is electronic resources so what software resources are we using to do this and when we are conducting the supplier .

Qualifications we might even have zoom meetings and things like that so there might be a little bit that you can look at without respect as to resources the fifth part of the process approach to auditing is who is doing the audit so with whom are we doing this supplier qualification process .

So um who's doing it on your side of the company who are the different people in your company what are the different levels are buyers involved is quality involved is an auditor involved and then you might also ask who's involved at the suppliers side you should also be asking step six .

With how is this done so what procedure is used for supplier qualification do you have a supplier quality management procedure like we do sys 11 is ours what is the form that you use so do you have an approve supplier approval form we have a approval form that we fill in do you have a log that you complete .

That would be another example for us it's a an actual log form that we fill in so those are different things you might even have a standard for certain processes not for supplier quality generally but um that could be something that you look at .

And then the last step step seven is metrics and a lot of people don't have metrics for processes but um how how much um how many suppliers will you evaluate before you approve one .

Um how many supplier audits do you do per year that might be another example um how many suppliers do you have some companies have too many suppliers and they don't have the right ratio of resources to the number of suppliers they need to shrink their supply chain and consolidate suppliers um we see that a lot some of the best .

Run companies i've ever seen they have a very small number of suppliers in fact one of the best quality systems i ever saw um it was actually a png facility uh procter gamble and that facility only had 25 suppliers and he can be this huge facility cranking out millions and millions of .

Products medical devices all the time and they only had 25 suppliers that was amazing to me that they could do all that work was just a short list and they actually had some redundancy on that list too they had more than one supplier for certain key components so that was really impressive to me so when you're doing this .

Audit or a mock fda inspection whatever you're doing you could be asking those seven questions that are part of the process approach step one through seven you could ask those is part of the qsed approach so when you get to supplier quality .

As one of the management process on your last day of doing an fda inspection or a mock fda inspection there's no reason why you can't ask those seven questions you don't also when you are doing an audit checklist and you're making sure they have that agreement i could still do um my seven steps my seven questions .

When i'm auditing yes they have a supplier quality agreement i gave you examples of all the things that we might ask so they're not mutually exclusive techniques but the process approach to auditing those seven questions they're so easy to .

Learn because there's only seven of them whereas the audit regulatory checklist that companies develop those should not be yes or no questions and unfortunately the gd the gd2 i think it was um or was it ge 210 gd210 i think was an .

Audit checklist that was developed for candy cass auditors and all the questions on that were yes or no questions and there was no real space on that checklist for somebody to write these notes there wasn't a lot of training that was provided that used best practices and it was only the very small handful of candy cast .

Auditors that went to canada and got trained to became the gas auditors that were trained on how you do these audits properly and what questions you asked and what information you're supposed to put in the report and they made it very clear to everybody that got qualified as a candy cass auditor that you're supposed to still use the process .

Approach you just go down the checklist and make sure that you included those elements somewhere in your notes when you use the process approach and if you didn't you got to go back and cover that gap so that's how the regulatory checklist was used it was more of a did i cover that and where in my notes can it be found .

But that's not how anybody's using these regulatory checklists they're using it as an actual audit checklist and they're not even using an audit checklist properly because they're not saying what they looked at and what they found they're just using it as a yes no check the box and that's not effective auditing .

So it's really important when you are using these techniques it's not that the the audit checklist is better or worse than the fda accuse it manual which is better or worse than the process approach to auditing it's that you could use all three .

Simultaneously if you have an audit checklist that has the elements that are in the qzip manual and you are using the process approach to make sure you're doing an effective audit of that process and then going back through your checklist making sure you didn't forget anything where is it in my notes when i .

Use the process approach and so that's exactly how i try to do this so i i've been out in california doing this audit and i've been making sure that as i go through this did i cover this element did i cover this element did i cover this element where in my notes is it covered but i don't ask questions off an .

Audit checklist and i don't flip it through the pages of the q-zip manual and make sure i'm following every single line-by-line in the q-zip manual i haven't ever seen an fda inspector do that either they follow audit trails just like a really good quality system auditor from a notified .

Body they follow those audit trails if they're not sure about how a company is controlling a process they will dig deeper and ask more open-ended questions how do you do that why do you do that when do you do that how many times do you do that how do you know you're supposed to do it that way those are the kinds of questions they .

Ask they keep on drilling down trying to get as much detail as they can until they're satisfied that process is going to be consistently performed and it's effective and the conclusion at the end of that section of your audit report should be this process appears to be effective or this process appears to be effective .

Except for the following non-conformity or this process appears to be effective however you should consider the following possible process improvements so those are the three possible outcomes you either have a non-conformity you have an ofi or you have everything's great and move on to the next process .

So for those of you that are wondering what the difference is b
etween the q-zip manual and the audit checklist like an ndsap checklist or your own internal audit checklist in the process approach to auditing if it's a process approach to auditing you have to be looking at what the inputs of the process are the outputs of the process .

The um the resources that are needed for the process who is doing the process what are the procedures and forms and logs what are the metrics for that process and then you follow the audit trails that that takes you down that's an effective audit and you can do that whether you're using an audit .

Checklist to make sure you cover relatory elements where is that found in my notes before i move on to the next area and you can also do that if you're following the general cues manual where it says i want to cover these four major areas the fda is going to say do you have a kappa procedure do you have a .

Capital law can i see it do you have kappa records but then they're going to drill down and ask tougher questions like um how do you make sure that your capitas are completed on time when you have a delay in a cap and you can't finish it as planned what do you do a lot of companies have extension .

Programs i hate them but that's what a lot of companies do how do you do that how do you control that to make sure that the process stays on schedule so these are just some examples for you um but don't think of these as three mutually exclusive methods .

There are three methods that can be used together if you really know your stuff well but you have to be not just an auditor but you have to be experienced in the area that you're auditing so part of the reason why notified body auditors um .

Are able to so quickly go through your facility and and complete an audit is because they spent five years in designing products for another medical device company before they were even allowed to work for a notified body and they got nine months of training on how to be an effective auditor day after day after day looking back at the at the iso .

Standard or the cmdr the canadian medical device regulations or the mdr for europe or looking at some some other standard they've looked at all these standards and they've learned them cold if you're doing audit of software you need a news no 62304 before you go do .

That audit if you're doing an audit of post market surveillance you should know the standard for post market surveillance iso 20416 so you need to know these standards really well and be able to make sure that the company's including those things in their process .

Uh as best practices i hope that was helpful for you though for those of you that weren't sure what the different methods are i'll put some links before they won't be right away with this post i'll add them later but i hope you enjoyed the live session and i'll see you next friday live bye