Scan your ML models with 4 lines of code

Quickly and easily scan, test and debug your ML models by using our Python library and testing server. We integrate with any Python model.
Open in Colab
import giskard
from
sklearn.pipeline import Pipeline
‚Äć
# Pipeline for the sklearn model
clf = Pipeline(...)
clf.fit(...)

‚Äć
# Wrap your Pandas DataFrame
dataset = giskard.Dataset(
  df=titanic_df, target=
"Survived"
‚Äć
)
‚Äć

# Wrap your model
model = giskard.Model(
  model=clf.predict_proba,
  model_type=
"classification"
‚Äć
)
‚Äć
# Scan for vulnerabilities
results = giskard.scan(model, dataset)
import giskard
from
transformers import AutoTokenizer
from
transformers import TFAutoModel
‚Äć
tk = AutoTokenizer.from_pretrained(...)
hf = TFAutoModel.from_pretrained(...)


# Pipeline for the model prediction
def pred_func(df):    
    return
softmax(hf(**tk(...).logits))

# Wrap your Pandas DataFrame and model
dataset = giskard.Dataset(df=text_df, ...)
model = giskard.Model(model=pred_func, ...)

# Scan for vulnerabilities
results = giskard.scan(model, dataset)
import giskard
from torchtext.models
  import
XLMR_BASE_ENCODER
‚Äć
model = XLMR_BASE_ENCODER.get_model(head=h)

# Pipeline for the model prediction
def pred_func(df):    
    
output_df = DataLoader(df.map(T))       return [model(i) for i in output_df]
‚Äć
# Wrap your Pandas DataFrame and model
dataset = giskard.Dataset(df=text_df, ...)
model = giskard.Model(model=pred_func, ...)

# Scan for vulnerabilities
results = giskard.scan(model, dataset)
import giskard

model = ... # tensorflow model

# Define a custom wrapper
class MyTensorFlowModel(giskard.Model):       def model_predict(self, df):
       return
self.model.predict(
           pipeline.transform(df))

# Wrap your Pandas DataFrame and model
dataset = giskard.Dataset(df, ...)
model = MyTensorFlowModel(model, ...)

# Scan for vulnerabilities
results = giskard.scan(model, dataset)
import giskard
from
langchain import chains, prompts
‚Äć
llm = ...
prompt = prompts.PromptTemplate(       input_variables=[
"product"],       template="..."
)
chain = chains.LLMChain(
   llm=llm,
   prompt=prompt
)


# Wrap your Pandas DataFrame and model
dataset = giskard.Dataset(df, ...)
model = giskard.Model(chain, ...)

‚Äć
# Scan for vulnerabilities
results = giskard.scan(model, dataset)
import giskard, requests
‚Äć
def pred_func(input_data):    
    
# Set up the API endpoint URL
   
api = "https://api.example.com/predict"
    # Send GET request to API & get     response
    response = requests.get(
    api, params={
"input": input_data}
   )
‚Äć
    
# Extract predictions from JSON     response
    return
...

# Wrap your Pandas DataFrame and model dataset = giskard.Dataset(df, ...)
model = giskard.Model(pred_func,   ...)


# Scan for vulnerabilities
results = giskard.scan(model, dataset)

Product workflow

Eliminate risks of biases, performance issues, and errors.
Become a ML superhero.

Integrate

API for any Python model, from tabular to LLMs: Torch, HuggingFace, Langchain, etc.

Scan

Automatically find vulnerabilities in your AI model directly in your notebook.

Test

Ready-made & extensible test suites, backed by State-of-the-Art ML research.

Automate

Protect your ML models against the risk of regressions, drift and bias.

Key features

Easy installation

Quickly install our Python library, compatible with any environment.

Model evaluation

Scan your model, from tabular to LLMs, with a few lines of code to instantly detect vulnerabilities.

Compliance with AI regulations

Avoid hefty fines for non-compliance by using our AI Quality Management System.

Secure & Fast

On-premise, with user authentification and encryption, so your data stays with you.

Collaboration

Create a human feedback loop with re-usable test components and visual debugging dashboards.

CI/CD integration

Automate and monitor your tests with each new model version to avoid regressions.

Supported Data and Model types

Data type

Tabular

LLMs, NLP

coming soon: Computer vision

Model type

Classification

Regression

Data

  • Tabular data
  • Text data
  • Images (coming soon)
  • Audio (coming soon)
  • Time¬†series¬†(coming¬†soon)¬†¬†¬†¬†¬†¬†¬†¬†¬†

ML model task

  • Classification
  • Regression
  • Text generation
  • Image generation (coming soon)
  • Time series forecasting (coming soon)

Integrates with your favorite Machine Learning tools

Join the community

This is an inclusive place where anyone interested in ML Quality is welcome! Leverage best practices from the community, contribute new tests, build the future of AI safety standards.