Auto generated mutations given model

[aminghadersohi]

Is there anything on the roadmap to create a class similar to SQLAlchemyObjectType but for mutations that would take a model in its Input and automatically create a graphene.Mutation subclass that has all the fields in it? The reason I ask is that I am thinking about brewing my own and if its not on the roadmap I'd be happy to contribute unless there is a good reason why this is bad. I think I can create a SQLAlchemyMutationMeta and SQLAlchemyMutation class to take the model from the Input class, fetch the fields in it and pass appropriate attrs to MutationMeta.

[marcovc]

Do you have any code to share for doing that? I just realized that graphene-sqlalchemy is completely useless to me if I have to duplicate my model definitions anyway (because of the mutations)...
Thanks

[aminghadersohi]

The SQLAlchemy Model/Store:

Base = sqlalchemy.ext.declarative.declarative_base()

class ClientRoleStore(Base):
    __tablename__ = "clientrole"
    uuid = Column(CHAR(36), primary_key=True, nullable=False, default=lambda: str(uuid1()))
    created_at = Column('created_at', DateTime, nullable=False,
                        server_default=func.now())
    updated_at = Column('updated_at', DateTime, index=True, nullable=False,
                        server_onupdate=func.now(),
                        server_default=func.now())
    deleted_at = Column('deleted_at', DateTime, nullable=True)
    name = Column(String(150), unique=True, nullable=False)
    description = Column(String(500))
    enabled = Column(Boolean, nullable=False, default=False)

The GraphQL object type:

class ClientRole(SQLAlchemyObjectType):
    class Meta:
        model = ClientRoleStore
        interfaces = (relay.Node,)

This is the main goods. Extend MutationMeta:

# just a util function
def camel_to_snake(s):
    s = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', s)
    s = re.sub('(.)([0-9]+)', r'\1_\2', s)
    return re.sub('([a-z0-9])([A-Z])', r'\1_\2', s).lower()

class SQLAlchemyMutationMeta(MutationMeta):
    @staticmethod
    def __new__(cls, name, bases, attrs):
        # Also ensure initialization is only performed for subclasses of
        # SQLAlchemyMutation
        if not is_base_type(bases, SQLAlchemyMutationMeta):
            return type.__new__(cls, name, bases, attrs)

        input_class = attrs.get('Input', None)
        if not input_class or not getattr(input_class, 'model', None) \
                or not getattr(input_class, 'field', None):
            return MutationMeta.__new__(cls, name, bases, attrs)

        assert is_mapped(input_class.model), (
            'You need to pass a valid SQLAlchemy Model in '
            '{}.Meta, received "{}".'
        ).format(name, input_class.model)

        field_name = camel_to_snake(input_class.field.__name__);

        inspected_model = inspect(input_class.model)

        def mutate(cls, instance, args, context, info):
            session = get_session(context)

            arg_attrs = {}
            for name, column in inspected_model.columns.items():
                # TODO perhaps ignore uuid, deleted_at, created_at and updated_at
                # as to not allow the user to ever set them manually
                arg_attrs[name] = args.get(name, None)

            field = input_class.model(**arg_attrs)
            # TODO: use param to determine create, update, delete. Right now just does create
            session.add(field)

            try:
                session.commit()
                ok = True
                message = "ok"
            except SQLAlchemyError as e:
                message = e.message
                ok = False

            kwargs = {
                'ok': ok,
                'message': message,
                field_name: field
            }
            return cls(**kwargs)

        input_attrs = {}

        # get the fields from the sqlalchemy model and map their types
        for name, column in inspected_model.columns.items():
            # TODO perhaps ignore uuid, deleted_at, created_at and updated_at
            input_attrs[name] = convert_sqlalchemy_column(column)
            # if the column has a the default don't require it
            if column.default or column.server_default:
                input_attrs[name].kwargs['required'] = False

        mutation_attrs = {
            'Input': type('Input', (object,), input_attrs),
            'ok': graphene.Boolean(),
            'message': graphene.String(),
            'mutate': classmethod(mutate),
            field_name: graphene.Field(input_class.field)
        }

        cls = MutationMeta.__new__(cls, name, bases, mutation_attrs)
        return cls


class SQLAlchemyMutation(six.with_metaclass(SQLAlchemyMutationMeta, Mutation)):
    pass

Define the graphql mutation object type using the class we just created above:

class CreateClientRole(SQLAlchemyMutation):
    class Input:
        model = ClientRoleStore
        field = ClientRole
        # we can pass another field here to indicate that its a create, update, or delete. Right now its just create

Define the mutation and query:

class Mutation(graphene.ObjectType):
    create_clientrole = CreateClientRole.Field()

class Query(graphene.ObjectType):
    clientroles = SQLAlchemyConnectionField(ClientRole)
   
schema = graphene.Schema(query=Query,
                         mutation=Mutation,
                         types=[ClientRole])

Now we can run this:

mutation {
  createClientrole(name: "role_name", description: "description", enabled: true) {
    ok
    message
    clientRole {
      uuid
      enabled
      name
      description
      createdAt
      deletedAt
      updatedAt
      id
    }
  }
}

that will return this:

{
  "data": {
    "createClientrole": {
      "message": "ok",
      "clientRole": {
        "description": "description",
        "enabled": true,
        "name": "role_name",
        "updatedAt": "2017-01-02T22:43:04",
        "deletedAt": null,
        "id": "Q2xpZW50Um9sZTplMDk2M2MzNS1kMTdmLTExZTYtYmYxMC1mNDVjODljYTVhM2Q=",
        "createdAt": "2017-01-02T22:43:04",
        "uuid": "e0963c35-d17f-11e6-bf10-f45c89ca5a3d"
      },
      "ok": true
    }
  }
}

[marcovc]

Wow, thanks! I'll try to use that (I'm still digesting it :)

[aminghadersohi]

let me know what you think.

[Marviel]

Bump. I would love to see this implemented.

[Fercho191]

How is the progress of this feature?

[Marviel]

@aminghadersohi thanks for the code! I've tried it out, and one thing I've had trouble with is adding in the field indicating the create, update, or delete. I keep getting this error: "Found different types with the same name in the schema: printertype_id, printertype_id"

This only seems to happen when I have two classes, CreatePrinter, and UpdatePrinter that both subclass SQLAlchemyMutation, and require the same model / field.

[qubitron]

Here's an example I was able to get working with graphene 2.0, with both create and edit operations:

import graphene
import graphene_sqlalchemy

from graphene.types.utils import yank_fields_from_attrs
from graphene.types.objecttype import ObjectTypeOptions
from graphene_sqlalchemy.types import construct_fields

class SQLAlchemyMutationOptions(ObjectTypeOptions):
    model = None # type: Model
    create = False # type: Boolean
    arguments = None  # type: Dict[str, Argument]
    output = None  # type: Type[ObjectType]
    resolver = None  # type: Callable

class SQLAlchemyMutation(graphene.Mutation):
    @classmethod
    def __init_subclass_with_meta__(cls, model=None, create=False, registry=None, only_fields=(), exclude_fields=(), **options):
        meta = SQLAlchemyMutationOptions(cls)
        meta.create = create
        meta.model = model
        
        arguments = yank_fields_from_attrs(
            construct_fields(model, registry, only_fields, exclude_fields),
            _as=graphene.Argument,
        )

        super(SQLAlchemyMutation, cls).__init_subclass_with_meta__(_meta = meta, arguments=arguments, **options)

    @classmethod
    def mutate(cls, self, info, **kwargs):
        session = get_session(info.context)

        meta = cls._meta
        model = None
        if meta.create == True:
            model = meta.model()
            session.add(model)
        else:
            model = session.query(meta.model).filter(meta.model.id == kwargs['id']).first()

        for key, value in kwargs.items():
            setattr(model, key, value)
        session.commit()
        return model

    @classmethod
    def Field(cls, *args, **kwargs):
        return graphene.Field(
            cls._meta.output, args=cls._meta.arguments, resolver=cls._meta.resolver
        )

class AddCustomer(SQLAlchemyMutation):
    class Meta:
        model = models.Customer
        create = True
        exclude_fields = ['id']
    Output = Customer

class EditCustomer(SQLAlchemyMutation):
    class Meta:
        model = models.Customer
    Output = Customer

class Mutations(graphene.ObjectType):
    addCustomer = AddCustomer.Field()
    editCustomer = EditCustomer.Field()

Note this currently assumes the ID field is named 'id'. To do a create operation, you have to use exclude_fields to remove the 'id' field or it will error that the id field is missing from the input arguments.

Another option is to automatically create an Input type based off of the SQL model and define your own mutate methods. Here's an example:

class SQLAlchemyInputObjectType(graphene.InputObjectType):
    @classmethod
    def __init_subclass_with_meta__(cls, model=None, registry=None, skip_registry=False,
                                    only_fields=(), exclude_fields=(), connection=None,
                                    use_connection=None, interfaces=(), id=None, **options):
        sqla_fields = yank_fields_from_attrs(
            construct_fields(model, registry, only_fields, exclude_fields),
            _as=graphene.Field,
        )
        # Add all of the fields to the input type
        for key, value in sqla_fields.items():
            setattr(cls, key, value)
        super(SQLAlchemyInputObjectType, cls).__init_subclass_with_meta__(**options)

class CustomerInput(SQLAlchemyInputObjectType):
    class Meta:
        model = models.Customer

class CustomerMutation(graphene.Mutation):
    class Arguments:
        customer = CustomerInput(required=True)
    Output = Customer
    def mutate(self, info, customer):
        print(customer)

class Mutations(graphene.ObjectType):
    printCustomer = CustomerMutation.Field()

I'm thinking about having the mutation also use the input type, but it's not clear it makes sense to do it that way.

[Toilal]

Thanks for your examples.

It seems to fail to go through a relationship though, because registry is None. (At least for the 1st example)

  File "/home/toilal/idea-projects/planireza/backend/planireza/__main__.py", line 4, in <module>
    run()
  File "/home/toilal/idea-projects/planireza/backend/planireza/app.py", line 27, in run
    import_sanic_modules()
  File "/home/toilal/idea-projects/planireza/backend/planireza/app.py", line 20, in import_sanic_modules
    from .endpoints import graphql
  File "/home/toilal/idea-projects/planireza/backend/planireza/endpoints/graphql.py", line 2, in <module>
    from ..graphql.schema import schema
  File "/home/toilal/idea-projects/planireza/backend/planireza/graphql/schema.py", line 42, in <module>
    class Mutations(graphene.ObjectType):
  File "/home/toilal/idea-projects/planireza/backend/planireza/graphql/schema.py", line 43, in Mutations
    createReference = CreateReference.Field()
  File "/home/toilal/idea-projects/planireza/backend/planireza/graphql/mutations.py", line 51, in Field
    cls._meta.output, args=cls._meta.arguments, resolver=cls._meta.resolver
  File "/home/toilal/idea-projects/planireza/backend/.venv/lib/python3.6/site-packages/graphene/types/field.py", line 53, in __init__
    self.args = to_arguments(args or OrderedDict(), extra_args)
  File "/home/toilal/idea-projects/planireza/backend/.venv/lib/python3.6/site-packages/graphene/types/argument.py", line 48, in to_arguments
    arg = arg.get_type()
  File "/home/toilal/idea-projects/planireza/backend/.venv/lib/python3.6/site-packages/graphene/types/dynamic.py", line 21, in get_type
    return self.type()
  File "/home/toilal/idea-projects/planireza/backend/.venv/lib/python3.6/site-packages/graphene_sqlalchemy/converter.py", line 32, in dynamic_type
    _type = registry.get_type_for_model(model)
AttributeError: 'NoneType' object has no attribute 'get_type_for_model'

[Toilal]

For now I understand that,

1. I need to exclude relationships with exclude_fields.
2. I need to implement import get_session.

I get this working too after excluding relationships.

I'll try to enhance those snippets to support automatically exclude relationships, and identifier on creation.

[Toilal]

Here it is : https://gist.github.com/Toilal/9e9d4eaaeb3ec7650e742e0484510977

So it works like @qubitron first example, but handles primary keys and relationships properly by introspecting the SQLAlchemy model.

At the moment, relationships are simply excluded, but it could be enhanced to support relationships by passing objects and array of objects in query parameters, and handle the recursion in the implementation.

[Toilal]

Pull request opened: #87.

class ReporterType(SQLAlchemyObjectType):
    class Meta:
        model = Reporter

class Mutation(graphene.ObjectType):
    createReporter = create(ReporterType)

query = '''
    mutation createReporter{
        createReporter (firstName: "ABC", lastName: "def") {
            firstName,
            lastName,
            email
        }
    }
'''

schema = graphene.Schema(mutation=Mutation)
# SQLAlchemy session must be given to context for mutations.
result = schema.execute(query, context_value={'session': session})

it's implemented with Mutation subclass types defined in mutation.py

[qubitron]

@Toilal yes, I ran into some troubles getting relationships to work but ended up with a solution that is workable enough for my project. One problem if you try to automatically generate input types for relationships, is that most relationships end up pointing back to themselves either directly or indirectly. For me this resulted in an infinite recursion of input types getting generated.

For it to work totally automatically you need to have a heuristic of not following back-props, or maybe somehow the type registry is supposed to alleviate this problem (I don't understand how to use it).

My simple solution for now was to only follow relationships if there is a corresponding input field explicitly declared on the input type.

So for example, in my case a customer can have a contact and I want to be able to specify this contact during add and edit:

class AddCustomerInput(SQLAlchemyInputObjectType):
    class Meta:
        model = models.Customer
        
    contact = graphene.Argument(AddContactInput)

During mutate I use the following code to look for relationships and instantiate fields for relationships that are explicitly listed on the input types:

            def setModelAttributes(model, attrs):
                relationships = model.__mapper__.relationships
                for key, value in attrs.items():
                    if key in relationships:
                        if getattr(model, key) is None:
                            # instantiate class of the same type as the relationship target
                            setattr(model, key, relationships[key].mapper.entity())
                        setModelAttributes(getattr(model, key), value)
                    else:
                        setattr(model, key, value)

            setModelAttributes(model, kwargs['input'])

I've modified my above code quite a bit as I've been making it work for my project, I've put my latest in the following gist (including example input types): https://gist.github.com/qubitron/6e7d48667a05120dc58bdfd0516b3980

[alexisrolland]

It would be great to have this feature. +1

[danaronson]

@Toilal I'm trying to use your code to automatically create an update mutation. However, my non-null fields become required. Obviously if I'm trying to update the other fields, I don't really need to require these. Any pointers how I can take the non-null types that get created for arguments and turn them into the non non-null types?

[flewellyn]

Is there any word on when this code may be merged?

[Toilal]

1. I think it should be enhanced with @qubitron and @danaronson comments.
2. I won't write those enhancements because I don't use GraphQL anymore at the moment.

[flewellyn]

That's fair enough. I'm looking at @qubitron's patch, and it looks like it may offer some additional flexibility. I will experiment with those classes.

[qubitron]

@flewellyn let me know if you have any feedback regarding the classes, I'm happy to make updates to them. I'd like to merge in my classes with @Toilal's PR but I haven't yet found the time.

[flewellyn]

Thank you, @qubitron . I had to make a few adjustments to get them to work, mostly in the area of missing imports. I also added a bit of smarts about omitting ID columns that are not just named "id", using SQLAlchemy's inspection. I have it omitting columns that are autoincrementing primary keys, or if they are timestamps with a default.

Once I made those adjustments, everything worked smoothly. The only thing I'd like to add is the ability to hook into the "resolver" equivalent for the mutation, the way you can write a resolver for the query. I have some preprocessing and validation I'd like to do.

[qubitron]

@flewellyn sorry about the missing imports! I had extracted this from my code. For validation and pre-processing I have an "on_before_commit" method that mutations can optionally override by defining e.g.:

    def on_before_commit(self, model, **kwargs):
        checkAccess(model)

I use this to do some pre-processing, but mainly for access checks. If the caller doesn't have access to mutate the object I raise an exception. I've updated my gist include this method.

[flewellyn]

Ahhh, I see. Nice trick!

Your implementation is only doing it for updates, though, and not for creation or deletion? Or am I misunderstanding the logic?

[qubitron]

@flewellyn ah that was a silly bug, thank you for finding that! I've updated the gist, the on_before_commit applies to all operations now.

[flewellyn]

No problem!

Your method is called after the mutation, to validate the created model. Is there a means to call it before, to validate the entire operation? I am guessing looking at the input type from the kwargs['input'] would do this. I shall experiment with it.

In any event, this looks really useful and powerful. Is this gist under MIT license, like the graphene_sqlalchemy codebase?

[qubitron]

Yep, using kwargs['input'] should allow you to get the unmodified input data, while the model variable gives you the created model. I figure it's nice to be able to access the model after it is created in case you want to simply add fields or modify the auto-created values. I've added the MIT license to the top of the gist so you can feel free to use this, and would be great if you could share your updates too!

One issue I've noticed is that raising an exception in this callback does not seem to propagate exception message to the response, I'm not sure how to do that.

[flewellyn]

Might need to reraise it? I'll experiment with that.

Meantime, here's my gist, where I made some changes to how automatic exclusion of fields in the input type works. Otherwise it's pretty much the same as yours.

https://gist.github.com/flewellyn/a655fb480abc24c6929385257286c3e5

[mhykgyver]

hi @flewellyn, thanks for sharing your code, I tested it and I have no errors/warnings but when I try to use Insomnia to test the mutation, I get this error:

{
	"errors": [
		{
			"message": "'Request' object has no attribute 'get'",
			"locations": [
				{
					"line": 2,
					"column": 2
				}
			]
		}
	],
	"data": {
		"addCalendarYear": null
	}
}

Any idea where I might have gone wrong or have I missed something in testing your code?

Here's a snippet of my code where I used yours:

import graphene

from models.unitOperationModels import CalendarYear as CalendarYearModel
from schemas.commonAttributes import UtilsAttributes
from utils.SQLAlchemyMutation import SQLAlchemyInputObjectType, \
    SQLAlchemyMutation

from graphene import relay
from graphene_sqlalchemy import SQLAlchemyObjectType


class CalendarYear(SQLAlchemyObjectType):

    class Meta:
        model = CalendarYearModel
        interfaces = (relay.Node, )


class AddCalendarYearInput(SQLAlchemyInputObjectType, UtilsAttributes):

    class Meta:
        model = CalendarYearModel

    year = graphene.String(default_value='')


class AddCalendarYear(SQLAlchemyMutation):

    class Arguments:
        input = graphene.Argument(AddCalendarYearInput)

    class Meta:
        model = CalendarYearModel
        create = True

    Output = CalendarYear

[flewellyn]

That looks like an exception being thrown from elsewhere in your call stack.

[mhykgyver]

hi @flewellyn,

I hope it's not too much to ask, but can you post a sample implementation on how you used your SQLAlchemyMutation version? Not much help from Stackoverflow and there's very limited resource in the internet for this implementation (this thread only, to be exact)...

TIA

[mhykgyver]

hi @flewellyn ,

you were right, it seems that I have to add db_session to

app.add_url_rule(
    '/graphql',
    view_func=GraphQLView.as_view(
        'graphql',
        schema=schema,
        graphiql=True,
        context={'session': db_session}
    )
)

and it worked for create...having issues though with updates... @qubitron, it seems that for updates at session.query

if meta.create:
            model = meta.model()
            session.add(model)
        # update
        else:
            model = session.query(meta.model).filter(meta.model.id == kwargs['id']).first()  #problem here

the meta.model.id is pointing to the Model's id and kwargs['id'] is pointing to graphene ID:

127.0.0.1 - - [13/Jun/2018 16:33:21] "?[37mPOST /graphql HTTP/1.1?[0m" 200 -
meta.model.id: CalendarYear.id
kwargs['id']: Q2FsZW5kYXJZZWFyOjE=
127.0.0.1 - - [13/Jun/2018 16:33:27] "?[37mPOST /graphql HTTP/1.1?[0m" 200 -

trying to fix it...

[mhykgyver]

found a fix as per 468 ... @resolve_only_args is deprecated though so I followed the v2.0 Upgrade Guide ... hope this helps for anyone who encountered what I encountered...

[flewellyn]

Minor bug in my implementation: if your SQLAlchemy model class defines an init method, it will not receive the arguments in the input type. I made a fix in my gist, passing in the input parameters to the constructor. This only affects creation, so update and delete mutations are not an issue.

https://gist.github.com/flewellyn/a655fb480abc24c6929385257286c3e5

[flewellyn]

@qubitron I've been trying to update my version of this code to work with relationships. I did as you documented, adding the relationship types as fields in the input type class. But they don't show up in the introspected schema, and I can't make use of them in mutations. Do you have any idea why this might be so?

[flewellyn]

I figured out the issue. I had to set the autogenerated relationship type on the metaclass, not on the class itself.

[Rafik-Belkadi]

i got this error when implementing your code :

TypeError: construct_fields() missing 1 required positional argument: 'connection_field_factory'

[Rafik-Belkadi]

class AddUserInput(SQLAlchemyInputObjectType):
    class Meta:
        model = UserModel
        

class CreateUser(SQLAlchemyMutation):
    class Arguments:
        input = graphene.Argument(AddUserInput)

    class Meta:
        model = UserModel
        create = True
        
    Output = User

[Agheb]

@qubitron what a lifesaver and it works flawlessly in my project! Thx for sharing this

[maquino1985]

First of all, thanks for this. For any new visitors, flask-graphql context doesn't work correctly anymore and a patch hasn't been merged to fix it or the documentation updated to reflect ti! so, if you don't have a session object in your info.context, head over to https://github.com/graphql-python/flask-graphql/pull/55/commits to fix it.

you'll need to add get_context to the GraphQLView and change the endpoint to this:

    '/graphql',
    'GraphQl',
    view_func=GraphQLView.as_view(
        'graphql',
        schema=schema,
        graphiql=True,  # for having the GraphiQL interface
        get_context=lambda: {'session': db_session}
    )
)

So, I'm having some trouble getting delete to work. It seems like it would be easy but I've seen suggestions like the one here: https://stackoverflow.com/questions/53597024/best-way-to-do-a-delete-operation-with-graphql-graphene but my app throws an error that Boolean type has no name attribute when I try that solution. If anyone could post a quick example I'd appreciate it. And are users of this code creating Edit, Create, and Delete Request mutation types for their graphene objects or is there a clever way to write fewer than 3 classes per mutation?

TYIA!

EDIT: I created a delete type like this, i'm not sure if there's a preferred way?

class DeleteRequest(SQLAlchemyMutation):
    class Arguments:
        id = Argument(UUID, default_value=None, name='id')

    class Meta:
        model = RequestModel
        delete = True

    Output = Request

[danaronson]

Any idea of how to get this merged?

[babaMar]

Is this going to be merged?

[CSGrandeur]

how's it going on?

[NJannasch]

@Rafik-Belkadi
I've solved it by importing:
from graphene_sqlalchemy.types import construct_fields, default_connection_field_factory, SQLAlchemyObjectType

and providing SQLAlchemyObjectType and default_connection_field_factory as an additional argument for:
construct_fields(SQLAlchemyObjectType, model, registry, tuple(only_fields), tuple(exclude_fields), default_connection_field_factory)

(or providing obj_type properly from the outside)

[himat]

Is there updated code for SQLAlchemyInputObjectType?
I'm getting an error in graphene_sqlalchemy/converter.py

    child_type = obj_type._meta.registry.get_type_for_model(relationship_prop.mapper.entity)
AttributeError: 'ObjectTypeOptions' object has no attribute 'registry'