New blog web-page

Blog has been moved to frazik

Unleashing the Potential: Modern Techniques to Boost Large Language Model Performance

Large Language Models (LLMs) have taken the AI world by storm, demonstrating remarkable capabilities in tasks ranging from text generation to code creation. But like any powerful tool, LLMs have room for optimization. Researchers are constantly exploring new techniques to push the boundaries of LLM performance and unlock their full potential. In this introductory blog post, we'll delve into some of the most promising approaches that are shaping the future of LLMs.

1. Retrieval-Augmented Generation (RAG):

Imagine an LLM that can not only generate text but also access and leverage relevant information from external sources. This is the core idea behind RAG models. RAG combines an LLM with a retrieval system that fetches information pertinent to the task at hand. The LLM then utilizes this retrieved information to enhance its generation process, leading to more factually accurate and informative outputs.

2. Chaining Transformers:

LLMs are often monolithic beasts, tackling entire tasks in one go. Chaining Transformers breaks down complex tasks into smaller, more manageable subtasks. Each subtask is handled by a specialized transformer model, and the outputs are sequentially chained together to achieve the final goal. This approach allows for more efficient training and potentially better performance on intricate tasks.

3. Prompt Engineering:

Think of prompts as instructions that guide an LLM towards the desired outcome. Prompt engineering focuses on crafting effective prompts that steer the LLM in the right direction. By carefully designing prompts that incorporate task-specific information and desired outcomes, researchers can significantly improve the quality and accuracy of LLM outputs.

4. Transfer Learning and Fine-tuning:

Pre-trained LLMs have learned a wealth of knowledge from massive datasets. Transfer learning and fine-tuning techniques leverage this pre-trained knowledge as a starting point for new tasks. By fine-tuning an LLM on a task-specific dataset, researchers can significantly reduce training time and improve performance compared to training from scratch.

This blog post has just scratched the surface of the exciting advancements in LLM technology. In future posts, we'll delve deeper into each of these techniques, exploring their specific applications and showcasing their potential to revolutionize various AI domains. 

Easy setup Kibana Nginx reverse proxy with Ansible

Motivation

Amazon OpenSearch Service cluster instance run inside a virtual private cloud. If you want to access Kibana dedicated to this instance you have two options. One is tunneling to EC2 bastion host which is realitvely straightforward. One of the disadvantages of this approach is that you need to share your bastion host keys to clients. Another is reverse proxy on bastion host to private OpenSearch Kibana. In this example, we are going to show how you can setup access to Kibana using Nginx reverse proxy and provision it with Ansible.

This example represent basic setup which can serve as basis for future improvements. This basis don't include secure access configuration (certifications, authentication). It uses HTTP between client and proxy server, for production environment using HTTPS is recommended in this context. It's an easier setup, but for other hand it's also less secure setup.

Example

Inventory

First you need to have inventory defined with one variable (open_search_endpoint) which should point to Kibana instance. Notice, we have two ec2 instances in our inventory. One can be for production environment, second for staging for example. 

	[ec2s]
	{host1} open_search_endpoint={open_search_endpoint1}
	{host2} open_search_endpoint={open_search_endpoint2}

view raw inventory hosted with ❤ by GitHub

Main playbook

Next we are going to define main Ansible playbook, which is pretty straight forward. For it to work, you need to have configuration files (default.config and ngnix.config) located in your path.

	# Can be run multiple times (idempotent), main playbook
	---
	- hosts: ec2s
	tasks:
	- name: Update all packages
	yum:
	name: "*"
	state: latest
	update_only: yes
	- name: Enable nginx for amazon linux 2
	shell: "amazon-linux-extras enable nginx1.12"
	become: yes
	- name: Install nginx
	yum:
	name: nginx
	state: latest
	- name: Delete existing dist folder
	file:
	path: "/etc/nginx/conf.d/default.conf"
	state: absent
	- name: Start nginx
	service:
	name: nginx
	state: started
	enabled: yes
	- name: Copy website default config
	copy:
	src: ../default.conf
	dest: /etc/nginx/conf.d/default.conf
	owner: root
	group: root
	mode: 0644
	- name: Copy nginx default config
	copy:
	src: ../nginx.conf
	dest: /etc/nginx/nginx.conf
	owner: root
	group: root
	mode: 0644
	- name: Set correct open search endpoint
	lineinfile:
	dest: /etc/nginx/conf.d/default.conf
	regexp: open_search_endpoint
	line: " proxy_pass https://{{open_search_endpoint}};"
	- name: Restart nginx
	service:
	name: nginx
	state: restarted

view raw main.yml hosted with ❤ by GitHub

Default configuration

For default.config, we are using simple proxy pass. If you want more secure connection, this is where you would configure HTTPS.

	server {
	listen 80 default_server;
	listen [::]:80 default_server;
	server_name localhost;
	location = / {
	rewrite ^ /_dashboards/ redirect;
	}
	location / {
	proxy_pass open_search_uri;
	proxy_set_header Authorization "";
	proxy_hide_header Authorization;
	}
	}

view raw default.conf hosted with ❤ by GitHub

Nginx configuration

	# For more information on configuration, see:
	# * Official English Documentation: http://nginx.org/en/docs/
	# * Official Russian Documentation: http://nginx.org/ru/docs/
	user nginx;
	worker_processes auto;
	error_log /var/log/nginx/error.log;
	pid /run/nginx.pid;
	# Load dynamic modules. See /usr/share/nginx/README.dynamic.
	include /usr/share/nginx/modules/*.conf;
	events {
	worker_connections 1024;
	}
	http {
	log_format main '$remote_addr - $remote_user [$time_local] "$request" '
	'$status $body_bytes_sent "$http_referer" '
	'"$http_user_agent" "$http_x_forwarded_for"';
	access_log /var/log/nginx/access.log main;
	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	include /etc/nginx/mime.types;
	default_type application/octet-stream;
	# Load modular configuration files from the /etc/nginx/conf.d directory.
	# See http://nginx.org/en/docs/ngx_core_module.html#include
	# for more information.
	include /etc/nginx/conf.d/*.conf;
	}

view raw nginx.conf hosted with ❤ by GitHub

Serverless React, AWS Lambda and API Gateway example

Here is small terraform example how you can create serverless React app that will use AWS API Gateway which will call AWS Lambda. 

API Gateway is proxying GET request to lambda. Lambda will simply return "Hello World". 

Actual terraform infrastructure definition. This terraform definition include CORS configuration.

	locals {
	function_name = "hello_world"
	handler = "index.handler"
	runtime = "nodejs14.x"
	zip_file = "hello_world.zip"
	}
	data "archive_file" "zip" {
	source_dir = "${path.module}/lambdas/hello-world"
	type = "zip"
	output_path = local.zip_file
	}
	resource "aws_lambda_function" "this" {
	description = "${var.config.team}-lambda-stream-es"
	// Function parameters we defined at the beginning
	function_name = local.function_name
	handler = local.handler
	runtime = local.runtime
	timeout = 15
	// Upload the .zip file Terraform created to AWS
	filename = local.zip_file
	source_code_hash = data.archive_file.zip.output_base64sha256
	// Connect our IAM resource to our lambda function in AWS
	role = var.config.es_lambda_role_arn
	}
	resource "aws_apigatewayv2_api" "this" {
	name = "${var.config.team}-${var.config.env}-lambda-gw"
	protocol_type = "HTTP"
	cors_configuration {
	allow_origins = ["https://www.first.com", "https://www.second.com"]
	allow_methods = ["GET"]
	allow_headers = ["content-type"]
	max_age = 300
	}
	}
	resource "aws_apigatewayv2_stage" "this" {
	api_id = aws_apigatewayv2_api.this.id
	name = "${var.config.team}-${var.config.env}-lambda-gw-stage"
	auto_deploy = true
	access_log_settings {
	destination_arn = aws_cloudwatch_log_group.this.arn
	format = jsonencode({
	requestId = "$context.requestId"
	sourceIp = "$context.identity.sourceIp"
	requestTime = "$context.requestTime"
	protocol = "$context.protocol"
	httpMethod = "$context.httpMethod"
	resourcePath = "$context.resourcePath"
	routeKey = "$context.routeKey"
	status = "$context.status"
	responseLength = "$context.responseLength"
	integrationErrorMessage = "$context.integrationErrorMessage"
	}
	)
	}
	}
	resource "aws_apigatewayv2_integration" "hello_world_integration" {
	api_id = aws_apigatewayv2_api.this.id
	integration_uri = aws_lambda_function.this.invoke_arn
	integration_type = "AWS_PROXY"
	integration_method = "POST"
	}
	resource "aws_apigatewayv2_route" "hello_world_route" {
	api_id = aws_apigatewayv2_api.this.id
	route_key = "GET /hello-world"
	target = "integrations/${aws_apigatewayv2_integration.hello_world_integration.id}"
	}
	resource "aws_cloudwatch_log_group" "this" {
	name = "/aws/api_gw/${aws_apigatewayv2_api.this.name}"
	retention_in_days = 30
	}
	resource "aws_lambda_permission" "api_gw" {
	statement_id = "AllowExecutionFromAPIGateway"
	action = "lambda:InvokeFunction"
	function_name = aws_lambda_function.this.function_name
	principal = "apigateway.amazonaws.com"
	source_arn = "${aws_apigatewayv2_api.this.execution_arn}/*/*"
	}

view raw serverless.tf hosted with ❤ by GitHub

Lambda which that will return "Hello World".

	// AWS lambda that call ECS service to get similar products
	module.exports.handler = async (event) => {
	console.log('Event: ', event);
	let responseMessage = 'Hello, World!';
	return {
	statusCode: 200,
	headers: {
	'Content-Type': 'application/json',
	},
	body: JSON.stringify({
	message: responseMessage,
	}),
	}
	}

view raw hello.js hosted with ❤ by GitHub

ALB rule based routing with terraform

From 2017 Amazon started supporting Host-based routing on Application Load Balancer. Content-based routing was supported before, so Application Load Balancer now supports both Host- and Path-based rules. You can combine both routing types to create complex rules for your services. If you are looking how to combine both routing types, please look at this stack overflow answer: https://stackoverflow.com/a/46304567.

In this example we are going to show you how you can use a single Application Load Balancer(ALB) for separate ECS’s. Imagine, you have 30 ECS’s running on Fargate. You want 10 of those to be exposed. If you would use ALB for every single ECS, this would be very inefficient and expensive. To combat this, AWS supports routing traffic on ALB to Target Groups (ECS can be one example of a target group) based on rules. In this way, you can have one ALB which will route traffic to the Target Group (in our example this will be ECS). 

Below is one example of how you can do that with comments about specific details.

	# First create a target group for each of your services, this is target group to
	# Docker ECS service running on AWS Fargate
	resource "aws_lb_target_group" "this" {
	name = "${local.this_name}-alb-tg"
	port = var.app_port
	protocol = "HTTP"
	vpc_id = var.config.vpc_id
	target_type = "ip"
	health_check {
	healthy_threshold = "3"
	interval = "120"
	protocol = "HTTP"
	matcher = "200-299"
	timeout = "119"
	path = var.health_check_path
	unhealthy_threshold = "2"
	}
	tags = {
	Team = var.config.team
	Environment = var.config.env
	Application = var.app_name
	}
	}
	# Then, create ALB. This is the internal ALB.
	# Hint: *Always use tags. It's much easier to do cost management and resource tracking.
	resource "aws_lb" "this" {
	name = "${local.wd_app_name}-alb"
	internal = true
	subnets = var.config.lb_subnets_ids
	load_balancer_type = "application"
	security_groups = [aws_security_group.lb.id]
	idle_timeout = 300
	tags = {
	Team = var.config.team
	Environment = var.config.env
	Application = local.wd_app_name
	}
	}
	# This is how you route traffic to a specific target group based on the host header.
	# For each of your services, you need to create specific rules for each target group.
	# In this example, we have only one target group and one router.
	resource "aws_lb_listener_rule" "this" {
	listener_arn = "${aws_lb_listener.this.arn}"
	action {
	type = "forward"
	target_group_arn = "${aws_lb_target_group.this.arn}"
	}
	condition {
	host_header {
	values = var.aws_lb_listener_rules
	}
	}
	}
	# This is the security group for ALB.
	# Be aware, this is a very permissive security group for internal ALB.
	# Tailor it to your needs.
	resource "aws_security_group" "lb" {
	name = "${local.this_name}-lb-sg"
	description = "Access to the Application Load Balancer (ALB)"
	vpc_id = var.config.vpc_id
	ingress {
	protocol = "tcp"
	from_port = 443
	to_port = 443
	cidr_blocks = ["0.0.0.0/0"] # All IP ranges
	}
	egress { # All traffic is allowed
	protocol = "-1" # -1 is equivalent to "All"
	from_port = 0
	to_port = 0
	cidr_blocks = ["0.0.0.0/0"]
	}
	tags = {
	Name = "${local.this_name}-lb-sg"
	Team = var.config.team
	Environment = var.config.env
	Application = var.app_name
	}
	}

view raw alb-rule-based-routing-with-terraform.tf hosted with ❤ by GitHub

React.js PDF export

If you need to eport React.JS components to PDF, this is one way how to do it. This approach uses html2canvas to generated canvas objects, then uses pdfMake to export them to PDF. You need to add ids to components you want to export:

	import React, {useState, Component} from 'react';
	import html2canvas from "html2canvas";
	import pdfMake from "pdfmake/build/pdfmake";
	import Button from '@material-ui/core/Button';
	const PdfExport = props => {
	const DOWNLOAD_FILE_NAME = "WebDesigner-Design.pdf"
	const printToPdf = () => {
	const { ids } = props;
	function nextStep(sections = [], i = 0) {
	if (i >= ids.length) {
	let pdfExportSetting = {
	content: sections
	};
	pdfMake.createPdf(pdfExportSetting).download(DOWNLOAD_FILE_NAME);
	return;
	}
	html2canvas(document.getElementById(ids[i])).then(canvas => {
	let data = canvas.toDataURL();
	let d = {
	image: data,
	width: 450
	}
	sections.push(d);
	nextStep(sections, ++i);
	});
	}
	nextStep();
	};
	return (
	<div>
	<Button onClick={printToPdf} style={{ backgroundColor: "green" }}>
	Print It
	</Button>
	</div>
	);
	};
	export default PdfExport;

view raw react-to-pdf.js hosted with ❤ by GitHub

Python script to find structure of opportunities on Slovakia job market

Introduction

I will try to answer this simple question using google colab + python notebook + web crawling Slovak job ad site + simple NLP (mainly using regex and simple text transformations) and pandas with sklearn.

It's impossible to answer this question using something like TIOBE programming index. This index is composed using trend searches in popular search engines. It doesn't take into consideration what is actual demand for some programming language on job market, let alone niche market like Slovak.

How is this possible?

This is possible now due to change of law on Slovak job market, which basically force companies to publish lowest possible salary they are willing to pay for position. Companies tends to put higher figures in ads, to compete with each other. So real salaries are bit higher, but it should average itself out. There is one problem tough. There is no regulation what type of salary they should put on ad, so there are companies that put net salary and other put gross salary. But. there are not so many of those that put gross salaries.

Data

For correctness, jobs with salary lower than 900 and bigger than 5500 EUR will be ignored, because there is higher probability they are false positive.

We will crawl most popular Slovak job ad site. Crawler will crawl through roughly 1200 pages of IT jobs. Some of which are full programming jobs, others are something in between (Managers, Support, Testers)

We will use corpus of words that will represent most popular programming languages. There will be tree different strategies for parsing programming languages from ad text. You shouldn't worry to much about this. Main reason for this it's difficulty to parse words like "C" or "R" programming languages from ad text, so we must treat it as single word that have no word boundaries.

Python scripts

Here is link to read only google colab python notebook without crawl code (code that actually rip/downloads content from job ad site)

Click here to see the scripts

Summary

As you can see there are some interesting surprises. Java is main language to learn if you want to make between 3000 and 4000 Euros.

Who knew bash is so important to learn? But on other hand is not so hard to learn it. :)

For lower paying positions PHP is main language, but you can also see there R at second spot (maybe some error in parsing?)

It no surprise that for higher salaries than 4000 EUR there is no clear winner. You must be generalist at these positions (Architects, Team Leads, Tech Leads). So answer to the question in title is: None, or there is not silver bullet, just be good at what you do and make sure to learn as much as you can.

Serverless Architectures

Serverless Architectures 

SA are new approach to application designs. This is hot topic in the software architecture world right now. All "Big Three" (Amazon, Google, Microsoft) are heavily investing in "Serverlessnes" right now.

What is it?

Unlike traditional architectures (where application run on server) it’s run in stateless compute containers that are event-triggered, ephemeral (may only last for one invocation), and fully managed by a third party. These are typically “rich client” applications—think single-page web apps, or mobile apps. These apps use vast ecosystem of cloud-accessible databases authentication services (e.g., Auth0, AWS Cognito), and so on. But if you need computation, you can use for example AWS Lambdas or Azure Functions.

Upsides and downsides

Upsides:

• No need for system administration (everything is handled by third party)
• Decreases complexity of your product
• Natively micro service architecture. 
• Lesser cost to scale.
• Elasticity - native scaling 
• Smaller development and operational costs
• Decrease time to market

Downsides:

• Debugging and monitoring is still and issue.
• Tooling is not there quite yet.
• Cold start issues.

Truffle execute external script call contract function

errors:

 * Error: VM Exception while processing transaction: out of gas
 * Error: Cannot create instance of YourContract; no code at address
 * Error: sender account not recognized
 * Error: invalid address

If you get some of these errors while trying to execute contract function from truffle script, here is proper way to do it:

	//First get all accounts on testrpc
	web3.eth.getAccounts((err, acc) => {
	let accounts = acc;
	contract.deployed().then(function(instance){
	//you need to specify from and gas parameters
	instance.yourFunction(someParam, {from: accounts[0], gas: 500000});
	}).then(function(result) {
	console.log(result);
	});
	});

view raw truffle-script-function.js hosted with ❤ by GitHub

Truffle external scripts working example

If you getting errors bellow while trying to execute your truffle external script continue to read.

errors:

TypeError: Cannot read property 'apply' of undefined 

exception at require.js:128:1 

TypeError: fn is not a function 

For some reason it was quite difficult to find solution how to run external script. After some time I finally have figure it out so I'm sharing it with world:

	module.exports = function(callback) {
	let Web3 = require('web3');
	const truffleContract = require('truffle-contract')
	let contract = truffleContract(require('../build/contracts/MetaCoin.json'));
	var provider = new Web3.providers.HttpProvider("http://localhost:8545");
	var web3 = new Web3(provider);
	contract.setProvider(web3.currentProvider);
	//workaround: https://github.com/trufflesuite/truffle-contract/issues/57
	if (typeof contract.currentProvider.sendAsync !== "function") {
	contract.currentProvider.sendAsync = function() {
	return contract.currentProvider.send.apply(
	contract.currentProvider, arguments
	);
	};
	}
	contract.deployed().then(function(instance){
	return instance.yourFunction();
	}).then(response => {
	console.log(response);
	});
	}

view raw truffle-extenal.js hosted with ❤ by GitHub