well, it was about time to start writing some code in this - supposedly - blog. time to get our hands dirty.
i want to talk about the latest stuff i have been playing with, which turns out to be testcontainers. and believe me, testcontainers are the real deal.
i used to hate writing integration tests. well, i still do - but testcontainers make it suck considerably less.
what are testcontainers anyway?
testcontainers are like that friend who shows up to help you move and actually does work instead of just eating your pizza. they spin up docker containers for your dependencies so you can run proper integration tests against actual infrastructure, not mocks or some sqlite pretending to be your production database.
the whole idea is simple but brilliant:
- spin up docker containers for your dependencies
- run your tests against real infrastructure
- tear everything down once tests are done
- act like nothing happened
it’s like the perfect crime, but for testing.
why should you even care?
maybe you’re thinking: “why would i bother with integration tests when i have perfectly good unit tests?”
let me tell you about the time i broke adidas’s entire product exchange system worldwide for a whole weekend. yes, actual adidas, the shoe company. yes, actual worldwide. and as always, the fix was a couple of lines of code.
unit tests wouldn’t have caught that. integration tests might have.
go ahead and write shit code, but at least do good tests.
integration tests are the ones that save you from getting that 3AM call from your boss. but they’re often such a pain to set up that we skip them or half-ass them.
testcontainers in action
for this demonstration, i’ll set up a basic golang API with a postgresql database, and test the interaction with it in my repository layer. we’ll keep it simple but practical.
domain entities
i define my entities in my domain layer:
internal/domain/dojo.go
type Dojo struct {
ID string `json:"id"`
Name string `json:"name"`
Address string `json:"address,omitempty"` // optional
Phone string `json:"phone,omitempty"` // optional
Email string `json:"email,omitempty"` // optional
FoundedDate time.Time `json:"foundedDate,omitempty"` // optional
}
func NewDojo(
name,
address,
phone,
email,
foundedDate string,
) (Dojo, error) {
if err := validation.Validate(name, validation.StringNotEmpty); err != nil {
return Dojo{}, err
}
var date time.Time
if strings.TrimSpace(foundedDate) != "" {
if err := validation.Validate(foundedDate, validation.ValidDate); err != nil {
return Dojo{}, err
}
date, _ = time.Parse(time.DateOnly, foundedDate)
}
return Dojo{
ID: uuid.NewString(),
Name: name,
Address: address,
Phone: phone,
Email: email,
FoundedDate: date,
}, nil
}
i like to differentiate between my domain entities and database models, but for this simple example the entity will suffice.
define the interface
in the same file:
internal/domain/dojo.go
var (
ErrCreateDojo = errors.New("coulnd't create dojo")
ErrUpdateDojo = errors.New("coulnd't update dojo")
ErrDeleteDojo = errors.New("coulnd't delete dojo")
ErrGetDojo = errors.New("coulnd't find dojo")
ErrListDojos = errors.New("coulnd't list dojos")
)
type DojoRepository interface {
Create(entity *Dojo) (*Dojo, error)
Update(ID string, entity *Dojo) (*Dojo, error)
Delete(ID string) error
Get(ID string) (*Dojo, error)
List(filter *DojoFilter) ([]*Dojo, error)
}
i define the interface that describes the interaction with any database we want to use. clean code principles dictate we code to interfaces, not implementations. sometimes i actually follow that advice.
implement the repository
initialize the repository:
internal/infra/postgresql/dojo.go
type DojoRepository struct {
db *gorm.DB
}
func NewDojoRepository(
db *gorm.DB,
) domain.DojoRepository {
return &DojoRepository{
db: db,
}
}
why am i using gorm? because as much as it sometimes gives me headaches, for simple querying it saves a lot of boilerplate. and i’m lazy. there, i said it.
let’s implement the interface methods:
func (instance *DojoRepository) Get(entityID string) (*domain.Dojo, error) {
var model *domain.Dojo
if err := instance.db.
Where("id = ?", entityID).
First(&model).
Error; err != nil {
e := fmt.Errorf("%v: %v", domain.ErrGetDojo, err)
logger.Error(e.Error())
return nil, e
}
logger.Debug(
"Fetched dojo: %+v", model)
return model, nil
}
i want my repository layer to already return a domain-defined error with context. sometimes i just log the error details here and return the domain error instead, but then you lose context when the error eventually bubbles up to the user.
func (repo *DojoRepository) List(
filter *domain.DojoFilter,
) ([]*domain.Dojo, error) {
var models []*domain.Dojo
if err := manageDojoFilters(repo.db, filter).
Order("Name").
Find(&models).
Error; err != nil {
e := fmt.Errorf("%v: %v", domain.ErrListDojos, err)
logger.Error(e.Error())
return nil, e
}
logger.Debug("Fetched %d dojos with %+v", len(models), filter)
return models, nil
}
i’ve been following an approach where i define a struct with the fields that can be filtered, then apply them programmatically when the values aren’t empty:
func manageDojoFilters(query *gorm.DB, filter *domain.DojoFilter) *gorm.DB {
if strings.TrimSpace(filter.Name) != "" {
query = query.Where("dojos.name = ?", filter.Name)
}
return query
}
the rest of the methods follow a similar pattern:
func (repo *DojoRepository) Create(entity *domain.Dojo) (
*domain.Dojo, error) {
if err := repo.db.Create(&entity).Error; err != nil {
e := fmt.Errorf("%v: %v", domain.ErrCreateDojo, err)
logger.Error(e.Error())
return nil, e
}
logger.Debug(
"Dojo created: %+v", entity)
return entity, nil
}
debug logging is great when you want to trace what’s happening, but keep it light so it doesn’t drown your logs in noise.
func (repo *DojoRepository) Update(
entityID string,
entity *domain.Dojo,
) (*domain.Dojo, error) {
var model *domain.Dojo
if err := repo.db.
Model(&model).
Where("id = ?", entityID).
Updates(entity).
Error; err != nil {
e := fmt.Errorf("%v: %v", domain.ErrUpdateDojo, err)
logger.Error(e.Error())
return nil, e
}
logger.Debug(
"Dojo updated: %+v", entity)
if err := repo.db.
Where("id = ?", entityID).
First(&model).
Error; err != nil {
e := fmt.Errorf("%v: %v", domain.ErrUpdateDojo, err)
logger.Error(e.Error())
return nil, e
}
return model, nil
}
fetching the updated record isn’t strictly necessary, but it comes in handy when your relationships get complex and you need to preload associations.
func (repo *DojoRepository) Delete(entityID string) error {
var model *domain.Dojo
if err := repo.db.
Where("id = ?", entityID).
Delete(&model).
Error; err != nil {
e := fmt.Errorf("%v: %v", domain.ErrDeleteDojo, err)
logger.Error(e.Error())
return e
}
logger.Debug(
"Dojo deleted: %+v", model)
return nil
}
i usually verify that the entity exists before calling the repository, so at this point i already know there’s something to delete.
testcontainers in action (for real this time)
now for the juicy part. first, install the packages:
go get github.com/stretchr/testify
go get github.com/testcontainers/testcontainers-go
go get github.com/testcontainers/testcontainers-go/modules/postgres
we’ll leverage testify’s suite to structure our tests:
SetupSuite()runs before all testsTearDownSuite()runs after all testsSetupTest()runs before each testTearDownTest()runs after each test
internal/infra/postgresql/dojo_test.go
type DojoRepositoryTestSuite struct {
suite.Suite
ctx context.Context
db *gorm.DB
dbContainer *postgres.PostgresContainer
dbConnectionString string
}
setting up the testcontainer:
func (suite *DojoRepositoryTestSuite) SetupSuite() {
suite.ctx = context.Background()
// spin up the postgres container
dbContainer, err := postgres.Run(
suite.ctx,
"postgres:16.4",
postgres.WithDatabase("mojodojo-test"),
postgres.WithUsername("postgres"),
postgres.WithPassword("postgres"),
testcontainers.WithWaitStrategy(
wait.
ForLog("database system is ready to accept connections").
WithOccurrence(2).
WithStartupTimeout(5*time.Second),
),
)
suite.NoError(err)
// get the connection string
connStr, err := dbContainer.ConnectionString(suite.ctx, "sslmode=disable")
suite.NoError(err)
db, err := gorm.Open(pg.Open(connStr), &gorm.Config{})
suite.NoError(err)
suite.dbContainer = dbContainer
suite.dbConnectionString = connStr
suite.db = db
sqlDB, err := suite.db.DB()
suite.NoError(err)
err = sqlDB.Ping()
suite.NoError(err)
}
and making sure it gets cleaned up:
func (suite *DojoRepositoryTestSuite) TearDownSuite() {
err := suite.dbContainer.Terminate(suite.ctx)
suite.NoError(err)
}
setting up the database for each test by running migrations:
func (suite *DojoRepositoryTestSuite) SetupTest() {
sqlDB, err := suite.db.DB()
suite.NoError(err)
driver, err := psqlMigrate.WithInstance(sqlDB, &psqlMigrate.Config{})
suite.NoError(err)
d, err := iofs.New(mojodojo.Migrations, "migrations")
suite.NoError(err)
m, err := migrate.NewWithInstance(
"iofs",
d,
"postgres",
driver,
)
suite.NoError(err)
err = m.Up()
suite.NoError(err)
}
and tearing it down after each test:
func (suite *DojoRepositoryTestSuite) TearDownTest() {
sqlDB, err := suite.db.DB()
suite.NoError(err)
driver, err := psqlMigrate.WithInstance(sqlDB, &psqlMigrate.Config{})
suite.NoError(err)
d, err := iofs.New(mojodojo.Migrations, "migrations")
suite.NoError(err)
m, err := migrate.NewWithInstance(
"iofs",
d,
"postgres",
driver,
)
suite.NoError(err)
err = m.Down()
suite.NoError(err)
}
this seems like a lot of code, but the idea is simple: spin up a container, run migrations before each test, tear down migrations after each test, and finally kill the container when we’re done.
in part 2, we’ll refactor these helpers for reuse across multiple test suites.
now let’s write an actual test:
func (suite *DojoRepositoryTestSuite) TestCreate() {
// ensure database is empty
var models []*domain.Dojo
result := suite.db.Find(&models)
suite.NoError(result.Error)
suite.Equal(0, len(models))
repo := NewDojoRepository(suite.db)
// create the entity using the constructor
entity, err := domain.NewDojo(
"my cool ass dojo",
"42 Wallaby Way",
"",
"p.sherman@crueldentists.com",
"2003/11/28",
)
suite.NoError(err)
created, err := repo.Create(&entity)
suite.NoError(err)
// ensure a new entity has been added
result = suite.db.Find(&models)
suite.NoError(result.Error)
suite.Equal(1, len(models))
suite.Equal(created.ID, models[0].ID)
suite.Equal(created.Name, models[0].Name)
suite.Equal(created.Address, models[0].Address)
suite.Equal(created.Phone, models[0].Phone)
suite.Equal(created.Email, models[0].Email)
suite.Equal(created.FoundedDate, models[0].FoundedDate)
}
and let’s test the update functionality:
func (suite *DojoRepositoryTestSuite) TestUpdate() {
// ensure database is empty
var models []*domain.Dojo
result := suite.db.Find(&models)
suite.NoError(result.Error)
suite.Equal(0, len(models))
// add an entity directly to the db
entity := &domain.Dojo{
ID: uuid.NewString(),
Name: "my cool ass dojo",
}
result = suite.db.Save(&entity)
suite.NoError(result.Error)
// ensure it was added
var model *domain.Dojo
result = suite.db.First(&model)
suite.NoError(result.Error)
suite.NotZero(model)
repo := NewDojoRepository(suite.db)
// update the entity
model.Name = "Sherman"
model.Email = "sherman.69@crueldentists.com"
updated, err := repo.Update(model.ID, model)
suite.NoError(err)
// verify the update
result = suite.db.First(&model)
suite.Equal(updated.ID, model.ID)
suite.Equal(updated.Name, model.Name)
suite.Equal(updated.Address, model.Address)
suite.Equal(updated.Phone, model.Phone)
suite.Equal(updated.Email, model.Email)
suite.Equal(updated.FoundedDate, model.FoundedDate)
}
i’ll leave the delete and list tests as an exercise for the reader (i’ve always wanted to say that - feels so academic).
finally, don’t forget to actually run the tests:
func TestDojoRepository(t *testing.T) {
suite.Run(t, new(DojoRepositoryTestSuite))
}
a word of warning for rancher desktop users
if you’re using rancher desktop like i am, you might run into issues with testcontainer initialization.
check out the rancher desktop documentation and make sure apache maven is installed for mvn verify to work.
for apple silicon users, there’s additional setup in the testcontainers documentation. because nothing can ever just work, right?
in conclusion
to sum it all up:
- testcontainers are genuinely awesome for integration testing
- they let you test against real infrastructure, not mocks
- the setup can seem verbose, but it’s worth the effort
- rancher desktop wants to make our lives harder (but we persist)
in part 2, i’ll refactor the test helpers, add testing for failure scenarios, and maybe, just maybe, add end-to-end testing as well. but no promises - i’m notoriously bad at estimating how much effort things take.
until then, remember: good tests might save you from breaking things worldwide. trust me, i know.