Technology Radar

Posted on by

A new edition of ThoughtWorks’ Technology Radar was published this week.

https://www.thoughtworks.com/content/dam/thoughtworks/documents/radar/2023/09/tr_technology_radar_vol_29_en.pdf

To no one’s surprise, AI is the big theme of this edition, I was however attracted by 2 items:

Mermaid

Mermaid is in the adoption quadrant.

https://mermaid.js.org/

In my current project at ING, we used documentation-as-code from the beginning, except that our choice is PlantUML. What I can comment on my experience, in 3 aspects:

  • Consistency: By treating documentation as code, it becomes easier to maintain consistency between the code and its documentation, especially when it comes to software architecture. We use C4 and when we make structural changes, the diagrams and the new code are versioned in the same PR, showing exactly the evolution of the system;
  • Collaboration: Between engineers is great because it sounds more natural to edit documentation in the same flow as the software is edited. Outside the world of engineers, there is a barrier as it is necessary to know the markup and a point-and-click interface is more intuitive. This is evident in the discussion of the software context, in which interaction with business colleagues is necessary;
  • Automation: This is where documentation-as-code shines. If the team culture encourages comments on your codebase, several diagrams can be automatically generated.

Complexity

It’s difficult for me to agree with the argument that we should embrace complexity in software development. Complexity is something that must be combated in our design, in our implementation, in our processes. Using Cynefin as guide, our goal is to transition from the complex to the complicated and, look, the article uses AI as an example of complexity but AI operates in the complicated, by using patterns and knowledge to deliver decisions and answers.

I remember Dumbledore saying to Harry: “Soon we must all face the choice, between what is right and what is easy”. An architect’s job, and all engineers are architects to some extent, is to resist the temptation of the easiest solution; It is common in these situations to introduce accidental complexity. If we are talking about essential complexity, ok, but we should still fight to reduce it. But I don’t blame Thoughtworks for the approach, Dijkstra once said “complexity sells better”.

https://www.thoughtworks.com/en-us/insights/blog/technology-strategy/why-embracing-complexity-real-challenge-software-today

Read the Radar, it’s always interesting, especially check out what’s close to the adoption level.

API Days

Posted on by

This week I was in London attending the ApiDays event. Once a year I participate in an external event, in addition to studying alone, it is important to meet professional colleagues and talk about what happens in the trenches. To my surprise, AI was not the central topic, I watched around 30 lectures and only 4 were specifically about AI. Two major themes at the event:

API Governance: Emphasis given to the API lifecycle: definition->design->development-> testing->publication->operation. What I’ll take home:

  • The importance of documentation is the central point of your API, whether consumed by developers or read by robots.
  • The role of patterns in design. This is a growing market and your consumers expect you to follow OpenAPI, AsyncAPI, Semantic Versioning, HTTP response codes, Protocol Buffers definition language.
  • Your operation must provide freedom of choice, do not assume a cloud provider, do not force a gateway, be open.

Democratization of APIs: Here there are 2 views that are converging, on the one hand experts say that we should develop APIs thinking about devops and gitops. This vision places great importance on the governance aspects mentioned in the previous item; In addition, this vision highlights interoperability and composability as essential attributes for modern APIs. On the other hand, if we look at the most common composition of companies, only 10% are in the technology area (Gartner research shown in one of the lectures); We have 49% end users and 41% classified as business technologists. These 41% create technology or analysis solutions based on the solutions that the IT areas provide. They coined the term ‘post-API economy world’ to embrace these people who should have easy access to APIs. The simpler it is, the easier it will be for innovative products to emerge from the available information provided. This second vision focuses on open and public APIs, ecosystems and marketplaces.

I will mention 2 tools that I tested at the event and found fantastic:

Superface.ai: OK, there are API catalogs, we know the concept of observability, how do we deal with APIs that are similar but have different formats? Think of wttr.in and OpenWeatherMap, both of which allow you to see if it’s going to rain in London today. But the similarity ends there, it is necessary to write code for each of them. Furthermore, if one of them is unavailable, you are responsible for routing the order. Superface deals with this: (1) you say what your contract is (2) Superface has autonomous agents that discover APIs consistent with your contract and maps your contract to the contract of the APIs it found. (ok, there’s AI here🙂)

Postman mocking servers: I often use Postman to test APIs, and I discovered that you can prototype your API in it. Design your methods, specify contracts from examples, and Postman creates documentation and an endpoint. When I wanted to think about an API, I used httpbin, this is much cooler.

Home Office

Posted on by

In 2008 I worked at Itaú in the IT for relationship with investment funds. I was part of a team responsible for fund performance reports, used by traders to guide their strategies. Our support was 24×7, especially during the night when these reports were generated. We discussed whether it would be possible for the engineer responsible for stand-by to work from home; our request was denied on the grounds that it was more productive to work in the bank’s infrastructure, where everything was available and where all the stand-by engineers were. 2020 came, Covid came and we know the result: we work very well in our homes. My experience and that of colleagues: we work better from our homes.

Zoom has grown exponentially precisely in this niche and last week we read that its CEO prefers work at the office. Eric Yuan points to speed of innovation and interaction as motivators, between the lines I read ‘our productivity needs to increase’, yeah, productivity again is the culprit. Also between the lines, I read that on Zoom the high command doesn’t trust people except when they’re around so they can see what people is doing. Microsoft showed this in the 2022 Pulse Report:
“The majority of employees (87%) report that they are productive at work (…)” but “85% of global business decision makers say that the shift to hybrid work has made it challenging to have confidence that people are being productive”.

I work in IT, this is the world that I know and about which I can give my opinion. Here, engineers are productive working from home and there are doubts in the C-level. McKinsey gave voice to this feeling and brought its idea 2 weeks ago: Yes, you can measure software developer productivity. Kent Beck wrote about this in his newsletter. Given that crisis of confidence that I mentioned, metrics are proposed to measure what I am doing. Kent is precise in his analysis, I agree: they confuse “activity” with “productivity”. Productivity is about delivering value and transforming your company.

The dichotomy of remote work vs. in-person work should not be the central discussion, trust and productivity are the central elements. And besides, do you want to go back to the office every day?

Architecture and AI

Posted on by

When I wrote my Master’s thesis, conceptualizing Creativity was essential and I used J.P. Guildford‘s ideas as a basis: creativity is the ability to exhibit creative behavior to a remarkable degree. He conceptualized creativity as a factor within a general theory of intelligence, involving the divergent thinking that could be developed through interaction between individuals and their environments. His proposal uses divergent cycles, which make it possible to create alternatives to a design problem from different perspectives, and convergent cycles, emphasizing the best option for the problem, with no room for ambiguity.

I’ve been revisiting this topic discussing on Reddit whether ChatGPT can be considered creative and more and more I tend to say “Yes”. Last month an architect suggested me the book “Architecture in the Age of Artificial Intelligence-An introduction to AI for architects“; after all, an architect must have original ideas, could it be replaced by ChatGPT? It’s a short book (180pp) and I recommend reading it, the first 2 chapters are a summary of techniques and how we got here in the field of Artificial Intelligence (but don’t expect anything in depth). The remaining chapters offer the view that knowledge workers like architects will be helped by AI tools but can also be overwhelmed by them. And that it is necessary to anticipate and understand where this happens and prepare. The reports about XKool are fascinating and this Guardian article has great pictures.In my reading, the book considers AI capable of being creative, something that Nick Cave’s latest newsletter dismantles. According to Nick, “ChatGPT is fast-tracking the commodification of the human spirit by mechanising the imagination”, sounds an anachronism to me.

I, as a software developer, also see myself as a knowledge worker and I am studying a lot to be ready. I have also been using my children and their friends to (try) to understand how the next generation sees the world in a few years. Both see a world dominated by AI and fearful of their place in this world. I tell them what I wrote above: be prepared.

Cypress, new kid on the block

Posted on by

Selenium has been my go-to tool for UI test automation for a while, and for good reason. It was the front-runner, proving its worth time and time again. I’ve used it extensively but lately, I’ve leaned towards Cypress. I made a course on Cypress (all of the exercises can be found in my GH), I’ve become thoroughly convinced that it presents a more efficient and reliable alternative.

When it comes to analysing the differences between Selenium and Cypress, a few key factors stand out. Selenium supports multiple languages including Java, Python, C#, and others while Cypress is purely JavaScript. This could initially seem limiting but considering most web applications are now JavaScript-based, Cypress becomes a natural choice. Selenium tests run outside the browser while Cypress runs directly inside the browser. This lets Cypress take control of the entire automation process including network traffic, timers, and even the loading of JavaScript code. It’s this inner control that promises stability – a promise Selenium often can’t keep due to its reliance on numerous third-party factors. The possibility to test your automation with debugger options in the browser is game-changer, I spend a lot of time understanding my scripts using the console, styles and network tabs, I am sure FE developers are all very familiar with them.

Cypress provides a simple and comprehensive API to interact directly with the DOM, allowing you to write simple, effective tests, take a look:

describe('DOM Interactions Test', function() {
  it('Fills and submits a form', function() {
    cy.visit('https://www.example.com/form') // Cypress loads the webpage

    cy.get('input[name="firstName"]').type('John') // Cypress finds the input field firstName and types 'John' into it
    cy.get('input[name="lastName"]').type('Doe') // Cypress finds the input field lastName and types 'Doe' into it

    cy.get('button[type="submit"]').click() // Cypress clicks the submit button
  })
})

Compare this with Selenium

WebDriver driver = new ChromeDriver();

driver.get("https://www.example.com/form"); // Open URL 

WebElement firstName = driver.findElement(By.name("firstName"));
firstName.sendKeys("John"); // Fill the firstName input

WebElement lastName = driver.findElement(By.name("lastName"));
lastName.sendKeys("Doe"); // Fill the lastName input

WebElement submitButton = driver.findElement(By.tagName("button")); 
submitButton.submit(); // Submit the form

Cypress let you interact with CSS selectors!

Another key advantage of Cypress is its ability to handle asynchronous operations very easily.

describe('Asynchronous Handling Test', function() {
  it('Waits for an element to be visible', function() {
    cy.visit('https://www.example.com') // Cypress loads the webpage

    cy.get('#asyncButton', { timeout: 10000 }) // Cypress waits for the element with id 'asyncButton' to become visible for up to 10 seconds
      .should('be.visible') // Asserts that the element should be visible
      .click() // Clicks the button once it's visible
  })
})

Dealing with API’s is also a breeze:

describe("Todo List API Interaction", () => {
  // Define the base URL of your API
  const apiUrl = "https://example-api.com";

  beforeEach(() => {
    // Intercept the API call and load a JSON payload for the response
    cy.intercept("GET", `${apiUrl}/todos`, { fixture: "todos.json" }).as("getTodos");
    // Visit the application or a specific page where the API call is made
    cy.visit("/todo-list");
  });

  it("should fetch todos from the API and display them", () => {
    // Perform any actions that trigger the API call (e.g., click a button to fetch todos)
    cy.get("#fetch-todos-btn").click();

    // Wait for the API call to complete and the response to be displayed
    cy.wait("@getTodos");

    // Assert that the correct API endpoint was called and the response was handled properly
    cy.get(".todo-item").should("have.length", 3); // Assuming the response contains 3 todo items
  });
});

What to me is the big advantage is my caveat: interacting with CSS selectors demands that you be careful to choose those that are least likely to change and break your tests. Still, I only plan to use Cypress for my UI tests.

Bloom Filters

Posted on by

One of my favorite data structures to use for efficient search operations is the Bloom filter, I just came across this cool demo, check it out.

Named after its creator, Burton Howard Bloom, a Bloom filter is a space-efficient probabilistic data structure designed to answer a simple question: “Is this item in the set?”. Unlike other data structures, a Bloom filter trades off accuracy for speed and space efficiency, meaning it might sometimes return false positives but never false negatives.

The Bloom filter works by using a bit vector of size ‘m’ and ‘k’ hash functions. When an element is added to the filter, it is passed through all ‘k’ hash functions, which return ‘k’ indexes to set to ‘1’ in the bit vector. To check whether an element is in the filter, the same ‘k’ hash functions are applied. If all ‘k’ indexes in the bit vector are ‘1’, the filter returns ‘yes’; otherwise, it returns ‘no’. A ‘yes’ answer can mean either the element is definitely not in the set (true negative) or it might be in the set (false positive), but a ‘no’ answer always means the element is not in the set (true negative). Note that when implementing, you should think carefully about how many hash functions to use and what the acceptable rate of false positives is.

Bloom filters are widely used in software applications where the cost of false positives is less critical than the benefits of speed and space efficiency. Some of these applications include spell checkers, network routers, databases, and caches. For instance, Google’s Bigtable uses Bloom filters to reduce the disk lookups for non-existent rows or columns, significantly improving its performance. Medium uses Bloom filters to avoid recommending articles a user has already read. I have a password generator application and use a Bloom Filter to check if a password has already been used.