GraphQL Configuration Generation with Tailcall

What is Configuration Generation?

Configuration generation is the process of automatically generating graphQL configurations from the various sources such as REST, gRPC and already existing GraphQL configuration files.

Why is it Hard to Write GraphQL Schemas by Hand?

Writing GraphQL schemas manually presents several challenges that can complicate and slow down the development process:

1. Complex API Responses:
   • Large and Detailed Responses: APIs often return extensive and intricate data, making it laborious to map these responses accurately to GraphQL types.
   • Nested Structures: Dealing with deeply nested JSON objects requires careful handling to ensure all relationships and data hierarchies are correctly represented in the schema.
2. Data Consistency:
   • Missing Properties: APIs can have inconsistent data where some items might lack certain properties, necessitating meticulous examination to define accurate types and optional fields.
   • Dynamic Data: Handling APIs with dynamic data fields adds another layer of complexity, requiring flexible and robust schema definitions to accommodate various data shapes.
3. Migration Efforts:
   • Manual Workload: Converting existing REST APIs or gRPC to GraphQL involves substantial manual effort, such as
     • Type and Schema Writing: Each endpoint must be meticulously mapped to corresponding GraphQL types and queries.
     • Type Merging: Identifying types that are similar in configuration and merging them into single type is tedious and time taking task and prone to errors.
     • Duplicate Type: Identifying and eliminating duplicate in the entire configuration is challenging especially for large scheams, to ensure a clean schema.
     • Type Naming: Inferring and naming types manually, which requires a deep understanding of the underlying data structures and their relationships.
   • Error-Prone Process: The manual creation of schemas increases the likelihood of errors, leading to potential issues in data fetching and integration.

These challenges highlight the need for automated tools, which streamline the process of generating GraphQL schemas, ensuring accuracy and efficiency while reducing the manual workload and error potential.

For more insights on why manual GraphQL schema writing is becoming obsolete, you can read this blog post by Tailcall.

Features

---

Effortless REST Integration

Tailcall simplifies GraphQL schema generation from REST APIs, supporting various request types and scenarios. Let's understand this through various examples.

1. Simple GET Request: In the following example, we demonstrate how to generate a GraphQL schema from https://jsonplaceholder.typicode.com/posts endpoint.

   This configuration allows Tailcall to fetch data from the specified endpoint and generate a GraphQL schema and save it to output path provided in configuration.

   JSON Config Format

   {
     "inputs": [
       {
         "curl": {
           "src": "https://jsonplaceholder.typicode.com/posts",
           "fieldName": "posts",
           "headers": {
             "Accept": "application/json",
             "secretToken": "{{.env.TOKEN}}"
           }
         }
       }
     ],
     "preset": {
       "mergeType": 1.0
     },
     "output": {
       "path": "./jsonplaceholder.graphql",
       "format": "graphQL"
     },
     "schema": {
       "query": "Query"
     }
   }

   YML Config Format

   inputs:
     - curl:
         src: "https://jsonplaceholder.typicode.com/posts"
         fieldName: "posts"
         headers:
           Accept: "application/json"
           secretToken: "{{.env.TOKEN}}"
   preset:
     mergeType: 1.0
   output:
     path: "./jsonplaceholder.graphql"
     format: "graphQL"
   schema:
     query: "Query"

   Let's understand the above configuration file.

   Input: Defines the API endpoints that the configuration interacts with. Each input specifies:

   • src: Specifies the endpoint URL (https://jsonplaceholder.typicode.com/posts) in this example.

   • fieldName: A unique name that should be used as the field name, which is then used in the operation type. In the example above, it's set to posts.

     important

     Ensure that each field name is unique across the entire configuration to prevent overwriting previous definitions.

   • headers: Optional section for specifying HTTP headers required for the API request.

     tip

     Never store sensitive information like access tokens directly in configuration files. Leverage templates to securely reference secrets from environment variables.

   Preset: We've applied only one tuning parameter for the configuration. let's understand it in short.

   • We've set mergeType to 1.0, which basically tells config generator to merge any two GraphQL types that are exactly similar.

     if you're interested in understanding preset's in detail head over to preset section.

   Output: Specifies where and in what format the output data should be saved.

   • path: Defines the output file path (in above example, it's ./jsonplaceholder.graphql).
   • format: Specifies the output format as GraphQL (in above example, it's graphQL).

   Schema: Specifies the name of the Query operation type, which is Query in this example.

   Generated GraphQL Configuration

   Generated GraphQL Configuration

   schema
     @server
     @upstream(
       baseURL: "https://jsonplaceholder.typicode.com"
     ) {
     query: Query
   }
   
   type Post {
     body: String
     id: Int
     title: String
     userId: Int
   }
   
   type Query {
     posts: [Post] @http(path: "/posts")
   }

   ---

2. Simple Post Request

   In the following example, we demonstrate how to generate a GraphQL schema from https://jsonplaceholder.typicode.com/posts endpoint which requires some request body in order to produce the response.

   This configuration allows Tailcall to make a POST request to the upstream API and retrieve the response to generate a GraphQL schema, which is then saved to the output path specified in the configuration.

   JSON Config Format

   {
     "inputs":[
       {
         "curl": {
           "src": "https://jsonplaceholder.typicode.com/posts",
           "method": "POST",
           "body": {
             "title": "Tailcall - Modern GraphQL Runtime",
             "body": "Tailcall - Modern GraphQL Runtime",
             "userId": 1
           },
           "headers": {
             "Content-Type": "application/json",
             "Accept": "application/json"
           },
           "isMutation": true,
           "fieldName": "createPost"
         }
       }
     ],
      "preset": {
       "mergeType": 1.0
     },
     "output":{
       "path":"./jsonplaceholder.graphql",
       "format":"graphQL"
     },
     "schema":{
       "mutation":"Mutation"
     }
   }

   YML Config Format

   inputs:
     - curl:
         src: "https://jsonplaceholder.typicode.com/posts/1"
         fieldName: "createPost"
         method: "POST"
         isMutation: true
         body:
           title: "Tailcall - Modern GraphQL Runtime"
           body: "Tailcall - Modern GraphQL Runtime"
           userId: 1
         headers:
           Accept: "application/json"
           Content-Type: "application/json"
   preset:
     mergeType: 1.0
   output:
     path: "./jsonplaceholder.graphql"
     format: "graphQL"
   schema:
     query: "Query"        

   Let's understand the above configuration file.

   Input: Defines the API endpoints that the configuration interacts with. Each input specifies:

   • src: Specifies the endpoint URL (https://jsonplaceholder.typicode.com/posts in this example).

   • fieldName: A unique name that should be used as the field name, which is then used in the operation type. In the example above, it's set to createPost.

     important

     Ensure that each field name is unique across the entire configuration to prevent overwriting previous definitions.

   • headers: Users can specify the required headers to make the HTTP request in the headers section.

     tip

     Never store sensitive information like access tokens directly in configuration files. Leverage templates to securely reference secrets from environment variables.

   • body: This property allows you to specify the request body for methods like POST or PUT. If the endpoint requires a payload, include it here.

   • method: Specify the HTTP method for the request (e.g. GET, POST, PUT, DEL). If not provided, the default method is GET. in above example, it's set to POST.

   • isMutation: This flag indicates whether the request should be treated as a GraphQL Mutation. Set isMutation to true to configure the request as a Mutation. If not specified or set to false, the request will be treated as a Query by default. in above example it's set to true.

   Preset: We've applied only one tuning parameter for the configuration. let's understand it in short.

   • We've set mergeType to 1.0, which basically tells config generator to merge any two GraphQL types that are exactly similar.

     if you're interested in understanding preset's in detail head over to preset section.

   Output: Specifies where and in what format the output data should be saved.

   • path: Defines the output file path (in above example, it's ./jsonplaceholder.graphql).
   • format: Specifies the output format as GraphQL (in above example, it's graphQL).

   Schema: Specifies the operation type. In this example, it's a Mutation operation with the name Mutation.

Generated GraphQL Configuration

Generated GraphQL Configuration

schema @server @upstream {
  mutation: Mutation
}

input PostInput {
  body: String
  title: String
  userId: Int
}

type Mutation {
  createPost(createPostInput: PostInput): Post
    @http(
      baseURL: "https://jsonplaceholder.typicode.com"
      body: "{{.args.createPostInput}}"
      method: "POST"
      path: "/posts"
    )
}

type Post {
  body: String
  id: Int
  title: String
  userId: Int
}

info

This flexible configuration approach allows you to adapt Tailcall for various HTTP methods by modifying key sections like method, body, isMutation and headers. Tailcall will handle generating the appropriate GraphQL schema based on the provided API interactions.

Effortless gRPC Integration

Tailcall simplifies the process of generating GraphQL schemas from gRPC. By specifying the proto file path, Tailcall parses it and generates the corresponding GraphQL types and queries within minutes.

1. gRPC Integration: In the following example, we demonstrate how to generate a GraphQL schema from a news.proto file.

   This configuration allows Tailcall to parse the proto file, generate a GraphQL schema and save it to the output path provided in the configuration.

   JSON Config Format

   {
     "inputs": [
       {
         "proto": {
           "src": "./news.proto"
         }
       }
     ],
     "preset": {
       "mergeType": 1.0
     },
     "output": {
       "path": "./jsonplaceholder.graphql",
       "format": "graphQL"
     },
     "schema": {
       "query": "Query"
     }
   }

   YML Config Format

     inputs:
       - proto:
         src: "./news.proto"
     preset:
       mergeType: 1.0
     output:
       path: "./jsonplaceholder.graphql"
       format: "graphQL"
     schema:
       query: "Query"

Let's understand the above configuration file.

Proto: Defines the path to the proto file that the configuration interacts with.

• src: Specifies the path to the proto file (./news.proto in this example).

Preset: We've applied only one tuning parameter for the configuration. let's understand it in short.

• We've set mergeType to 1.0, which basically tells config generator to merge any two GraphQL types that are exactly similar.

  if you're interested in understanding preset's in detail head over to preset section.

Output: Specifies where and in what format the output data should be saved.

• path: Defines the output file path (in above example, it's ./jsonplaceholder.graphql).
• format: Specifies the output format as GraphQL (in above example, it's graphQL).

Schema: Specifies the name of the Query operation type, which is Query in this example.

Generated GraphQL Configuration

Generated GraphQL Configuration

schema @link(src: "./news.proto", type: Protobuf) @server {
  query: Query
}

type News @tag(id: "news.News") {
  id: Int
  title: String
  content: String
  author: String
}

type Query {
  news: [News] @grpc(method: "news.NewsService.GetNews")
}

for more insights on how gPRC works with GraphQL, you can read this GraphQL over gRPC article.

Hybrid Integration (REST + gRPC)

The Configuration Generator with Tailcall supports a hybrid integration of REST and gRPC. This feature allows you to leverage the strengths of both REST APIs and gRPC to create a unified GraphQL schema. By integrating both sources, you can ensure that your GraphQL schema is comprehensive and up-to-date with your existing APIs and data definitions.

Example Configuration

Here is an example configuration that demonstrates how to set up a hybrid integration using a REST and gRPC:

JSON Config Format

{
  "inputs": [
    {
      "curl": {
        "src": "https://jsonplaceholder.typicode.com/posts",
        "fieldName": "posts"
      }
    },
    {
      "proto": {
        "src": "./news.proto"
      }
    }
  ],
  "preset": {
    "mergeType": 1.0
  },
  "output": {
    "path": "./output.graphql",
    "format": "graphQL"
  },
  "schema": {
    "query": "Query"
  }
}

YML Config Format

inputs:
  - curl:
      src: "https://jsonplaceholder.typicode.com/posts"
      fieldName: "posts"
  - proto:
      src: "./news.proto"
preset:
  mergeType: 1.0
output:
  path: "./output.graphql"
  format: "graphQL"
schema:
  query: "Query"

Let's understand the above configuration file.

Inputs

• curl - section where we can specify the REST endpoint.
  • src: The URL of the REST API endpoint.
  • fieldName: The field name to use in the GraphQL schema for the REST data.
• proto - section where we can specify the Proto File.
  • src: The path to the Proto file.

Preset: We've applied only one tuning parameter for the configuration. let's understand it in short.

• We've set mergeType to 1.0, which basically tells config generator to merge any two GraphQL types that are exactly similar.

  if you're interested in understanding preset's in detail head over to preset section.

Output: Specifies where and in what format the output data should be saved.

• path: Defines the output file path (in above example, it's ./jsonplaceholder.graphql).
• format: Specifies the output format as GraphQL (in above example, it's graphQL).

Schema: Specifies the name of the Query operation type, which is Query in this example.

schema
  @link(src: "./news.proto", type: Protobuf)
  @upstream(baseURL: "https://jsonplaceholder.typicode.com")
  @server {
  query: Query
}

type News @tag(id: "news.News") {
  id: Int
  title: String
  content: String
  author: String
}

type Post {
  body: String
  id: Int
  title: String
  userId: Int
}

type Query {
  posts: [Post] @http(path: "/posts")
  news: [News] @grpc(method: "news.NewsService.GetNews")
}

Understanding Presets

This section is optional and can be used to generate a more optimized configuration by applying various transformers that improve the config generation process, such as automatically inferring meaningful names of the types, merging duplicate types, removing unused types, and more. If you find that the generated GraphQL configuration is sufficient for your needs, you can skip this section.

The config generator provides a set of tuning parameters that can make the generated configurations more readable by reducing duplication and making configuration more readable. This can be configured using the preset section present in configuration.

JSON

{
   "preset": {
    "mergeType": 0.8,
    "consolidateURL": 0.8,
    "treeShake": true,
    "unwrapSingleFieldTypes": true,
    "inferTypeNames": true,
  }
}

YML

preset:
  mergeType: 0.8
  consolidateURL: 0.8
  treeShake: true
  unwrapSingleFieldTypes: true
  inferTypeNames: true

Let's understand how each of the parameter works.

mergeType

This setting merges types in the configuration that satisfy the threshold criteria. It takes a threshold value between 0.0 and 1.0 to determine if two types should be merged or not. The default is 1.0. MergeType also supports union types as well as interface types but merging of these types will happen only when they match exactly.

Example 1: following types T1 and T2 are exactly similar, and with a threshold value of 1.0, they can be merged into a single type called M1:

Merging type T1 and T2 into M1

Merging type T1 and T2 into M1

# BEFORE
type T1 {
    id: ID
    firstName: String
    lastName: String
}

type T2 {
    id: ID
    firstName: String
    lastName: String
}

# AFTER: T1 and T2 are merged into M1.
type M1 {
    id: ID
    firstName: String
    lastName: String
}

Example 2: following types T1 and T2 are similar with a threshold value of 0.5, they can be merged into a single type called M1:

Merging type T1 and T2 into M1

Merging type T1 and T2 into M1

# BEFORE
type T1 {
    id: ID
    firstName: String
    age: Int
}

type T2 {
    id: ID
    firstName: String
    lastName: String
}

# AFTER: T1 and T2 are merged into M1.
type M1 {
    id: ID
    firstName: String
    lastName: String
    age: Int
}

Example 3: following types T1 and T2 are similar with a threshold value of 0.5 but we can't merge them as they have same field name but different types:

Can't Merge type T1 and T2 as they've same field name but different type

Can't Merge type T1 and T2 as they've same field name but different type

# BEFORE
type T1 {
    id: ID
    firstName: String
    age: Int
}

type T2 {
    id: ID
    firstName: String
    age: Float
}

Example 4: following types Foo and Bar will be merged into type M1 as they match exactly and same change will reflected in union type FooBar.

Merging type Foo and Bar into M1

Merging type Foo and Bar into M1

# BEFORE
type Foo {
    id: ID
    firstName: String
    age: Int
}

type Bar {
    id: ID
    firstName: String
    age: Int
}

union FooBar = Foo | Bar

# After merging

type M1 {
    id: ID
    firstName: String
    age: Int
}

union FooBar = M1

Example 5: following types Foo and Bar won't be merged into type M1 as they don't match exactly.

Can't Merge type T1 and T2 as they've same field name but different type

Can't Merge type T1 and T2 as they've same field name but different type

# BEFORE
type Foo {
    id: ID
    firstName: String
    age: Float
}

type Bar {
    id: ID
    firstName: String
    age: Int
}

union FooBar = Foo | Bar

---

consolidateURL

The setting identifies the most common base URL among multiple REST endpoints and uses this URL in the upstream directive. It takes a threshold value between 0.0 and 1.0 to determine the most common endpoint. The default is 0.5.

For example, if the Query type has three base URLs, using the consolidateURL setting with a 0.5 threshold will pick the base URL that is used in more than 50% of the http directives, http://jsonplaceholder.typicode.com, and add it to the upstream, cleaning the base URLs from the Query type.

schema
  @server(hostname: "0.0.0.0", port: 8000)
  @upstream(httpCache: 42) {
  query: Query
}

type Query {
  post(id: Int!): Post
    @http(
      baseURL: "http://jsonplaceholder.typicode.com"
      path: "/posts/{{.args.id}}"
    )
  posts: [Post]
    @http(
      baseURL: "http://jsonplaceholder.typicode.com"
      path: "/posts"
    )
  user(id: Int!): User
    @http(
      baseURL: "http://jsonplaceholder.typicode.com"
      path: "/users/{{.args.id}}"
    )
  users: [User]
    @http(
      baseURL: "http://jsonplaceholder-1.typicode.com"
      path: "/users"
    )
}

After enabling the consolidateURL setting:

schema
  @server(hostname: "0.0.0.0", port: 8000)
  @upstream(
    baseURL: "http://jsonplaceholder.typicode.com"
    httpCache: 42
  ) {
  query: Query
}

type Query {
  post(id: Int!): Post @http(path: "/posts/{{.args.id}}")
  posts: [Post] @http(path: "/posts")
  user(id: Int!): User @http(path: "/users/{{.args.id}}")
  users: [User]
    @http(
      baseURL: "http://jsonplaceholder-1.typicode.com"
      path: "/users"
    )
}

---

unwrapSingleFieldTypes

This setting instructs Tailcall to flatten out types with single field.

for example:

type Query {
  foo: Foo
}

# Type with only one field
type Foo {
  bar: Bar
}

# Type with only one field
type Bar {
  a: Int
}

After setting unwrapSingleFieldTypes to true:

type Query {
  foo: Int
}

This helps in flattening out types into single field.

---

treeShake

This setting removes unused types from the configuration. When enabled, any type that is defined in the configuration but not referenced anywhere else (e.g., as a field type, union member, or interface implementation) will be removed. This helps to keep the configuration clean and free from unnecessary definitions.

Before applying treeShake, the configuration might look like this.

Before applying treeShake, the configuration might look like this.

type Query {
  foo: Foo
}

type Foo {
  bar: Bar
}

# Type not used anywhere else
type UnusedType {
  baz: String
}

type Bar {
  a: Int
}

After enabling treeShake, the UnusedType will be removed.

After enabling treeShake, the UnusedType will be removed.

type Query {
  foo: Foo
}

type Foo {
  bar: Bar
}

type Bar {
  a: Int
}

---

inferTypeNames

The inferTypeNames setting aims to enhance type naming consistency and readability by suggesting meaningful type names derived from its usage and shape.

Heuristic Algorithm

This is the default algorithm used to infer the name of the types in the configuration.

1. Generates Type Names: Creates type names from field names using pluralization and other heuristics.
2. Updates Configuration: Replaces existing type names with the inferred names and updates all references.

Before enabling inferTypeNames setting

Before enabling inferTypeNames setting

type T1 {
  id: ID
  name: String
  email: String
  post: [T2]
}

type T2 {
  id: ID
  title: String
  body: String
}

type Query {
  users: [T1] @http(path: "/users")
}

• User: Derived from T1, since T1 is linked to user data through the users field in the Query type. The new name User clearly indicates the type represents user information.

• Post: Derived from T2, since T2 is linked to post data through the post field within User. The new name Post clearly indicates the type represents post information.

After enabling inferTypeNames setting

After enabling inferTypeNames setting

type User {
  id: ID
  name: String
  email: String
  post: [Post]
}

type Post {
  id: ID
  title: String
  body: String
}

type Query {
  user: User @http(path: "/users")
}

By leveraging field names to derive type names, the schema becomes more intuitive and aligned with the data it represents, making it easier to understand and maintain.

Additional Considerations:

• Priority Handling: Types directly associated with root operations are given higher priority during inference. For example, if T2 were associated with a root query or mutation type, it might have a higher priority for inference compared to other types.

• Pluralization Rules: The inferred type names are converted to singular form to align with typical GraphQL naming conventions. For instance, a type derived from a plural field name like comments would be singularized to Comment.

LLM Powered Inference

This is a more advanced completely opt-in feature. Sometimes it's not possible to infer names correctly based on usage, or a name is not available because its been used already. In such scenarios we leverage LLMs that understand relationships between fields, their schema and other meta information to infer type names. To allow Tailcall to connect to LLMs, you need to provide the API key in the configuration file.

JSON

{
  "llm": {
    "model": "gpt-4o",
    "secret": "{{env.LLM_API_KEY}}"
  }
}

YML

llm:
  model: "gpt-4o"
  secret: "{{env.LLM_API_KEY}}"

tip

Checkout our LLM section to get a list of all the LLM models that Tailcall supports.

Best Practices

When setting up your configuration file for GraphQL generation with Tailcall, consider these key parameters to optimize and customize your setup:

1. Merge Type: Controls the merging of similar GraphQL types to reduce duplication. Adjust the threshold (0.0 to 1.0) based on how strictly you want types to match for merging. the closer the number to 1.0, you get the best type inference in graphQL playground. Recommended threshold is anything above 0.9.

   JSON

   {
     "preset": {
         "mergeType": 0.9
     }
   }

   YML

   preset:
       mergeType: 0.9

2. Consolidate URL: Identifies the most common base URL among multiple REST endpoints and uses it in the @upstream directive. Set a threshold (0.0 to 1.0) to determine when to consolidate URLs. Recommended threshold is anything above 0.5.

   JSON

   {
     "preset": {
       "consolidateURL": 0.5
     }
   }

   YML

   preset:
       consolidateURL: 0.5

3. Headers: Never store sensitive information like access tokens directly in configuration files. Leverage templates to securely reference secrets from environment variables.

   JSON

   {
     "headers": {
       "secretToken": "{{.env.TOKEN}}"
     }
   }

   YML

   headers:
       secretToken: "{{.env.TOKEN}}"

FAQ's

Q. Can I use environment variables in my configuration?

Answer: Yes, you can use environment variables to securely reference sensitive information like access tokens. Here is an example:

JSON

{
  "curl": {
    "src": "https://jsonplaceholder.typicode.com/posts/1",
    "fieldName": "post",
    "headers": {
      "secretToken": "{{.env.TOKEN}}"
    }
  }
}

YML

curl:
  src: "https://jsonplaceholder.typicode.com/posts/1"
  fieldName: "post"
  headers:
      secretToken: "{{.env.TOKEN}}"

Q. How do I merge similar types in the configuration?

Answer: Adjust the mergeType parameter in the preset section to control the merging of similar types. A threshold value between 0.0 and 1.0 determines if two types should be merged or not. if you to understand this in detail then please head over to preset section. Here is an example:

JSON

{
  "preset": {
    "mergeType": 0.9
  }
}

YML

preset:
    mergeType: 0.9

Q. What if I have multiple REST endpoints with different base URLs?

Answer: Use the consolidateURL parameter to identify the most common base URL among multiple REST endpoints and it will automatically select the most common base url and add it to the @upstream directive. Here is an example:

JSON

{
  "preset": {
    "consolidateURL": 0.5
  }
}

YML

preset:
    consolidateURL: 0.5

Q. Can I specify multiple input sources in a single configuration?

Answer: Yes, you can specify multiple input sources, such as different REST endpoints or Proto files, in a single configuration. Here is an example:

JSON

{
  "inputs": [
    {
      "curl": {
        "src": "https://jsonplaceholder.typicode.com/posts",
        "fieldName": "posts"
      }
    },
    {
      "proto": {
        "src": "./news.proto"
      }
    }
  ],
  "schema": {
    "query": "Query"
  }
}

YML

inputs:
  - curl:
      src: "https://jsonplaceholder.typicode.com/posts"
      fieldName: "posts"
  - proto:
      src: "./news.proto"
schema:
  query: "Query"