Some features might depend on packages which are not installed by default!
A growing list of optional dependencies can be found here.
Arch Linux
sudo pacman -S texlive-most python-pygments
Usage
With Python 3 and LaTeX installed you can easily compile your project using the maketex script which simplifies the compilation progress, handles multiple source files and removes unnecessary files.
For most use-cases you only have to run ./maketex which compiles the main.tex file using pdflatex while looking for bibliography and glossary entries.
Latex
If (for some reason) you do not want to depend on the maketex script you can also use pdflatex, makeglossaries and bibtex from the shell.
pdflatex -shell-escape main # Initial compilation
makeglossaries main # Compile glossaries
pdflatex -shell-escape main # Progressive compilation for glossaries
bibtex main # Compile bibliography
pdflatex -shell-escape main # Progressive compilation for bibtex
pdflatex -shell-escape main # Progressive compilation for bibtex
TexStudio
In TexStudio a custom command can be added under Options → Configure TexStudio → Build → User Commands. The following line completely compiles a LaTeX file with glossaries, bibliography and minted.
Of course you can also add the maketex script as a user command but you might want to set -lm so TexStudio can find your log files and minted cache after cleanup.
Overleaf is a popular online latex editor and is also fully supported by this template. Just download the archived repository or latest release and upload as a new project.
Options
Options are added to the \documentclass command usually found in main.tex.
\documentclass[<option1>,<option2>,...]{protocol}
Option
Result
en
Set main document language to english
landscape
Change the page format to landscape orientation
minted
Add and configure minted package
natbib
Change bibtex backend to natbib
nobib
No bibliography
nofonts
No additional fonts
noglo
No acronyms and glossary
nologos
No logos on titlepage
notable
No table on titlepage
notitle
No titlepage
notoc
No table of contents
parskip
Skip a line instead of indenting after blank line
sans
Load sans-serif fonts
Variables
Variables are set as commands with their parameter being the variable value.
A Dockerfile has been included to facilitate running the appplication.
$> cd docker
$> docker build -t ubuntu-apparatus .
$> docker run -it ubuntu-apparatus /bin/bash
$> cd /tmp
Usage
For addition:
$> ./apparatus XI IX
SVMMA: XX
One can add more than one number together.
$> ./apparatus VI IX LX
SVMMA: LXXV
For subtraction:
$> ./apparatus -s XI IX
SVMMA: II
One can subtract more than one number from the initial number.
$> ./apparatus -s XXX I IV X
SVMMA: XV
A note on forms
The apparatus supports Alternative forms of Roman Numerals as described in Wikipedia:
The “standard” forms described above reflect typical modern usage rather than a universally
accepted convention. Usage in ancient Rome varied greatly and remained inconsistent in
medieval and modern times.
Roman inscriptions, especially in official contexts, seem to show a preference for additive
forms such as IIII and VIIII instead of (or even as well as) subtractive forms such as IV and
IX. Both methods appear in documents from the Roman era, even within the same document.
“Double subtractives” also occur, such as XIIX or even IIXX instead of XVIII. Sometimes V and
L are not used, with instances such as IIIIII and XXXXXX rather than VI or LX.
This Chrome extension, designed with the latest Chrome updates Manifest V3, allows you to Enable Context Menu, Allow Copy and Right-click functionalities on websites that have disabled these commands. Additionally, it allows users to copy text from images using the OCR function.
Enable Copy and Highlight: Effortlessly copy text and highlight content on web pages.
Re-enable Context Menu: Regain access to the right-click context menu for a seamless browsing experience.
Remove Copytext Protection: Bypass restrictions set by websites to prevent copying text.
Absolute Mode: A robust mode to remove various types of protection against advanced methods disabling copy-paste functionality.
📣 How To Use
Begin by installing the Absolute Enable Copy & Image Reader extension from the respective browser store.
After installation, click on the extension icon located in the top-right corner of your browser.
In the popup, select between three options: “Enable Copy,” “Absolute Mode,” or “Image Reader (OCR).”
Post selecting your desired mode, start copying text from any website seamlessly.
📣 How I Built This Extension!
Recently, we published our first Chrome extension that enables right-click and copy functionality on websites that have disabled the text copy selection command on their websites. Additionally, we added an OCR Image Reader feature that allows you to extract text from images. In this article, we will show you how to create a Chrome extension that enables right-click functionality.
Here is a simple and detailed step-by-step guide on how you can create your own ‘Enable Right-Click & Copy’ Chrome extension.
Before starting the actual development of the ‘Absolute Enable Copy & Image Reader’ Chrome extension, we first need to understand what a Chrome extension is. Chrome extensions are small tools or software pieces that can be installed in browsers to add extra features, enhancing our browsing experience.
To make a simple extension you need simple web technologies like HTML, CSS, and JavaScript, along with some knowledge of web development. You will be pleased to know that our straightforward “Absolute Enable Copy & Image Reader” Chrome extension is built using these technologies.
Creating a Chrome extension is similar to creating a web application, but it requires a manifest.json file which we will discuss in this post.
Have you ever tried to copy important text from a website but couldn’t because it disabled the copy and right-click functions?
You can solve this with a handy browser extension that lets users right-click and copy, even on restrictive sites. Want to upgrade your browser experience? Here’s a simple step-by-step guide to making this extension from scratch.
1. Setting Up Your Workspace
First, you need a special folder (or “directory”) where all your files will live.
📂 Create a New folder named “Extension Name” on your computer.
Inside “Extension Name“, make these folders:
‘js‘ (for code stuff)
‘images‘ (for pictures)
‘css‘ (for style stuff)
This organization ensures that our files are well-structured and easy to locate.
2. The Heart: Manifest File
Every extension begins with a manifest.json file. This is like an ID card for your extension. This crucial file provides metadata about the extension: its name, version, permissions, and more.
First of all, we need the VS Code editor where we will write our HTML, CSS, or JavaScript code. So, download Visual Studio Code and install it on your PC or laptop. Now
Open Visual Studio Code on Your PC.
Go To File > Open Folder > Select “Extension Name” folder
Go To File > New File > Create a Manifest.json File
After creating a Manifest.json file, paste this HTML boilerplate code:
This manifest file is structured with the extension’s name, version, and description. It also includes a default page called popup.html and a background service worker named background.js. Furthermore, it specifies the necessary permissions, such as:
ActiveTab: This permission allows the extension to access the currently active tab in the browser.
Storage: This permission allows the extension to use local storage to store and retrieve data.
scripting: It grants the extension the ability to modify the behavior of web pages by injecting scripts into them.
You can have a look here to see all configurations of a manifest.json file.
Note
Place the manifest.json file at the root of your directory. This file will tell the browser about the core properties of your extension.
3. The Brain: Background Scripts & Page
To make the magic happen, you require two files: background.js and background.html. These files work in the background and handle different tasks without being noticed. They manage how the browser works.
The background scripts take care of tasks like managing a list of websites and enabling specific features, such as copying and right-clicking, on those websites.
Go To File > New File > Create a background.js File
After creating a background.js file, paste this JavaScript boilerplate code:
let websites_List = [];
chrome.storage.local.get(['websites_List'], function(value) {
websites_List = value.websites_List || [];
});
chrome.runtime.onMessage.addListener(function(request, sender, sendResponse) {
let text = request.text;
chrome.tabs.query({ currentWindow: true, active: true }, function(tabs) {
if (tabs[0] && tabs[0].url) { // Check if tabs[0].url is not undefined or null
let url;
try {
url = (new URL(tabs[0].url)).hostname;
} catch (error) {
console.error('Error constructing URL:', error, 'from:', tabs[0].url);
return;
}
state(url, text);
enableCopy(url, text, tabs[0].id);
}
});
if (text === 'delete-url') {
deleteUrl(request.url);
}
});
chrome.tabs.onUpdated.addListener(function(tabId, changeInfo, tab) {
if (changeInfo.status === 'complete' && tab.url) { // Check if tab.url is not undefined or null
let hostname;
try {
hostname = (new URL(tab.url)).hostname;
} catch (error) {
console.error('Error constructing URL:', error, 'from:', tab.url);
return;
}
inject(tabId, hostname);
}
});
function state(url, text) {
if (text === 'state') {
if (websites_List.indexOf(url + '#c') !== -1) {
chrome.runtime.sendMessage({
c: 'true'
});
}
if (websites_List.indexOf(url + '#a') !== -1) {
chrome.runtime.sendMessage({
a: 'true'
});
}
}
}
function enableCopy(url, text, tabId) {
if (text === 'c-true') {
websites_List.push(url + '#c');
inject(tabId, url);
saveData();
}
if (text === 'c-false') {
let index = websites_List.indexOf(url + '#c');
if (index > -1) {
websites_List.splice(index, 1);
saveData();
}
}
if (text === 'a-true') {
websites_List.push(url + '#a');
inject(tabId, url);
saveData();
}
if (text === 'a-false') {
let index = websites_List.indexOf(url + '#a');
if (index > -1) {
websites_List.splice(index, 1);
saveData();
}
}
}
async function inject(tabId, url) {
if (url !== undefined && url !== null) {
if (tabId !== undefined) {
if (websites_List.indexOf(url + '#c') !== -1) {
try {
await chrome.scripting.executeScript({
target: { tabId: tabId },
files: ['js/enable.js']
});
} catch (error) {
console.error('Error:', 'url:', url, '- tabId:', tabId, '\n', error);
}
}
if (websites_List.indexOf(url + '#a') !== -1) {
try {
await chrome.scripting.executeScript({
target: { tabId: tabId, allFrames: true },
files: ['js/enableA.js']
});
} catch (error) {
console.error('Error:', 'url:', url, '- tabId:', tabId, '\n', error);
}
}
}
}
}
function deleteUrl(url) {
let index = websites_List.indexOf(url);
if (index !== -1) {
websites_List.splice(index, 1);
saveData();
}
}
function saveData() {
chrome.storage.local.set({
'websites_List': websites_List
});
}
This JavaScript represents the logic of our background.js file. Now, let’s create a background.html file where the JavaScript code will be executed.
Place the background.js file (containing the logic) and the background.html file (containing the structure) in the root directory.
4. User Interaction: Popup Page & Script
Now that we’ve put the heart and brain into our extension, it’s time to introduce the popup. This is where users can see and interact with our extension. The popup acts as a small interactive window, crafted using a combination of HTML and JavaScript.
This page directly interacts with our background.js, which we created earlier to make the extension function. To enable our extension to work with the popup, we need two files: popup.html (for the appearance) and popup.js (for the behavior).
Go To File > New File > Create a popup.js File
After creating a popup.js file, paste this JavaScript boilerplate code:
let c = false;
let a = false;
let r = false;
chrome.runtime.sendMessage({
text: 'state'
});
chrome.runtime.onMessage.addListener(function(request) {
if (request.c === 'true') {
c = true;
}
if (request.a === 'true') {
a = true;
}
state();
});
document.querySelector('.enable-copy').onclick = function () {
enableCopy();
};
document.querySelector('.abs-mode').onclick = function () {
CopyRightPlusMode();
};
document.querySelector('.reload').onclick = function () {
chrome.tabs.reload();
window.close();
};
document.querySelector('.settings').addEventListener('click', function() {
chrome.tabs.create({
url: 'src/options.html'
});
window.close();
});
document.addEventListener('dragstart', function(e) {
e.preventDefault();
return false;
});
chrome.tabs.query({active: true, currentWindow: true}).then(function(tabs) {
let url = tabs[0].url;
if (!/^https?:\/\//i.test(url)) {
document.querySelector('.enable-copy').remove();
document.querySelector('.abs-mode').remove();
document.querySelector('.description').remove();
document.querySelector('.state').style = 'color: #a98e8e; height: 150px; display: grid; align-items: center;';
document.querySelector('.state span').innerHTML = 'Unable to run on this page';
}
});
function enableCopy(message) {
if (c === false) {
c = true;
message = {
text: 'c-true'
};
chrome.runtime.sendMessage(message);
} else {
c = false;
r = true;
message = {
text: 'c-false'
};
chrome.runtime.sendMessage(message);
}
state(r);
}
function CopyRightPlusMode(message) {
if (a == false) {
a = true;
message = {
text: 'a-true'
};
chrome.runtime.sendMessage(message);
} else {
a = false;
r = true;
message = {
text: 'a-false'
};
chrome.runtime.sendMessage(message);
}
state(r);
}
function state(r) {
if (c === true) {
document.querySelector('.enable-copy img').src = "https://github.com/Prepphint/images/on.png";
} else {
document.querySelector('.enable-copy img').src = 'images/off.png';
if (r == true)
reload();
}
if (a === true) {
document.querySelector('.abs-mode img').src = "https://github.com/Prepphint/images/on.png";
} else {
document.querySelector('.abs-mode img').src = 'images/off.png';
if (r == true)
reload();
}
if (c === false && a === false) {
document.querySelector('.state span').innerHTML = 'Not Enabled';
} else {
document.querySelector('.state span').innerHTML = 'Enabled';
}
}
function reload() {
document.querySelector('.reload').style.display = 'flex';
}
const git = document.getElementById("earnings");
git.onclick = () => { chrome.tabs.create({ url: "https://technomare.com/" }); };
After creating the popup.js file, it’s time to create the popup.html file where users can interact with our extension.
In this popup.html, we have set two modes of enabling copy.
“Enable Copy” Mode: This mode lets you copy text on smaller websites.
“Absolute Mode: This mode helps you to copy text forcefully on larger websites, even when their copy function is blocked.
We have also added some options like reloading the active page and have given access to the extension’s settings page to make the extension more interactive. but as of now, we haven’t used CSS to make our UI user-friendly and more good-looking. To make it better, you can give the popup a nicer look using some CSS code. to do this first, we create a new file “popup.css” with CSS code, Now
Go To File > New File > Create a popup.css File. (Make sure that this file is located in the “css” folder.)
This CSS file contains styles for the main page of a Chrome extension. It defines the appearance and layout of various elements on the popup page.
5. Add Some Extra Features: Options Page & Script
Next, let’s create an Options page. This page will display a list of websites where the extension can be used to enable the copy function, allowing users to select text from those sites. Users will also have the capability to delete Websites from this list.
Additionally, the Options page will fetch and display the latest posts from the RSS feed of our website, technomare.com. so for this, We’ll create three files: Options.js (for the behavior), Options.html (for the structure), and Options.css (for styling).
Go To File > New File > Create an Options.js File
After creating an Options.js file, paste this JavaScript boilerplate code:
(function() {
function callback(u) {
u = document.querySelector('#user-list');
chrome.storage.local.get('websites_List', function(value) {
if (value.websites_List !== undefined) {
for (var i = 0; i < value.websites_List.length; i++) {
getData(u, value.websites_List[i]);
}
empty(u);
} else {
return;
}
});
}
function getData(u, url, mode) {
var hostname = url;
var d = document.createElement('div');
u.appendChild(d);
d.className = 'table-row';
if (url.indexOf('#c') !== -1) {
url = url.replace('#c', '');
urlFilter = url + '##enable-copy';
mode = 'enable-copy';
} else {
url = url.replace('#a', '');
urlFilter = url + '##CopyRightPlus-mode';
mode = 'CopyRightPlus-mode';
}
d.innerHTML = `
<div class="row-label" url=${url} mode=${mode} >${urlFilter}</div>
<i class="row-delete" name=${mode} title="Delete"></i>
`;
d.querySelector('.row-delete').addEventListener('click', function() {
chrome.runtime.sendMessage({
text: 'delete-url',
url: hostname
});
d.remove();
empty(u);
});
}
function empty(u) {
var empty = document.querySelector('.list-empty');
if (empty !== null && u.querySelectorAll('.table-row')[0] !== null) {
empty.style.display = 'none';
}
if (u.querySelector('.table-row') === undefined || u.querySelector('.table-row') === null) {
empty.style.display = 'block';
}
}
window.onload = function() {
callback();
fetch('https://amureborn.com/feed/')
.then(response => response.text())
.then(str => {
let parser = new DOMParser();
let xmlDoc = parser.parseFromString(str, 'text/xml');
const items = xmlDoc.querySelectorAll('item');
const latestItems = Array.from(items).slice(0, 3);
let html = '';
latestItems.forEach(el => {
const title = el.querySelector('title').textContent;
const link = el.querySelector('link').textContent;
html += `
<div class="post">
<h2><a href="https://github.com/Prepphint/${link}" target="_blank">${title}</a></h2>
</div>
`;
});
document.getElementById('rss-feed').innerHTML = html;
})
.catch(error => {
console.error('Error fetching RSS feed:', error);
});
};
})();
After creating the Options.js file, let’s proceed to develop the Options.html file, where users can interact with our extension’s additional features.
Go To File > New File > Create an Options.html File
After creating an Options.html file, paste this html boilerplate code:
This code defines the appearance of various elements on the Options page, but it needs CSS to look nice. Let’s make an Options.css file to style our Options.html page.
Go To File > New File > Create a popup.css File. (Make sure that this file is located in the “css” folder.)
This CSS code provides styles for the header, tables, list items, footer, and other sections of the Options page. The goal is to achieve a neat and organized layout with consistent colors, fonts, and spacing.
6. Make The Extension Work ( Main JavaScript )
Congratulations! You’ve created almost all the necessary extension files. However, our main functions aren’t set up yet. As mentioned earlier, I’ve added two modes: the first is “Enable Copy” and the second is “Absolute Mode”.
These modes determine how our extension will operate. Let’s create a new file named enable.js For the “Enable Copy” mode, and another file named enableA.js for the “Absolute Mode”. (Make sure that those files are located in the “js” folder.)
Go To File > New File > Create an enable.js File.
After creating an enable.js file, paste this JavaScript boilerplate code:
(function() {
'use strict';
var css = document.createElement("style");
var head = document.head;
head.appendChild(css);
css.type = 'text/css';
css.innerText = `* {
-webkit-user-select: text !important;
-moz-user-select: text !important;
-ms-user-select: text !important;
user-select: text !important;
}`;
var elements = document.querySelectorAll("*");
for (var i = 0; i < elements.length; i++) {
if (elements[i].style.userSelect == 'none') {
elements[i].style.userSelect = 'auto';
}
}
document.oncontextmenu = null;
document.onselectstart = null;
document.ondragstart = null;
document.onmousedown = null;
document.body.oncontextmenu = null;
document.body.onselectstart = null;
document.body.ondragstart = null;
document.body.onmousedown = null;
document.body.oncut = null;
document.body.oncopy = null;
document.body.onpaste = null;
var doc = document;
var body = document.body;
var docEvents = [
doc.oncontextmenu = null,
doc.onselectstart = null,
doc.ondragstart = null,
doc.onmousedown = null
];
var bodyEvents = [
body.oncontextmenu = null,
body.onselectstart = null,
body.ondragstart = null,
body.onmousedown = null,
body.oncut = null,
body.oncopy = null,
body.onpaste = null
];
setTimeout(function() {
document.oncontextmenu = null;
}, 2000);
[].forEach.call(['copy', 'cut', 'paste', 'select', 'selectstart'], function(event) {
document.addEventListener(event, function(e) {
e.stopPropagation();
}, true);
});
})();
Next, create an enableA.js file for the “Absolute Mode”.
Go To File > New File > Create an enableA.js File.
After creating an enableA.js file, paste this JavaScript boilerplate code:
(function() {
'use strict';
var css = document.createElement("style");
var head = document.head;
head.appendChild(css);
css.type = 'text/css';
css.innerText = `* {
-webkit-user-select: text !important;
-moz-user-select: text !important;
-ms-user-select: text !important;
user-select: text !important;
}`;
[].forEach.call(['contextmenu', 'copy', 'cut', 'paste', 'mouseup', 'mousedown', 'keyup', 'keydown', 'drag', 'dragstart', 'select', 'selectstart'], function(event) {
document.addEventListener(event, function(e) {
e.stopPropagation();
}, true);
});
})();
After creating those files, let’s create one final file named “injection.js” for our extension which will inject both enable.js and enableA.js functions with our extension. (Make sure that this file is also located in the “js” folder.)
Go To File > New File > Create an injection.js File.
After creating an injection.js file, paste this JavaScript boilerplate code:
This JavaScript will ensure that either “enable.js” or “enableA.js” is properly injected into sites to override certain events (like context menu, copy, cut, paste, etc.) that may be preventing the user from right-clicking or copying content.
7. Important: Upload All Required Images
Be careful when uploading images for the extension. All required images should be organized and uploaded within a single folder, making it easier to locate the image paths. Earlier, we created a folder named “Images” inside the “Extension Name” folder. Put all the needed images there.
Create the necessary images using Canva or any other image creation or editing tool you’re comfortable with.
8. Adding Your Files To Chrome://Extensions
Once all of this is done, we’re ready to add this project as an extension to our Chrome browser.
To do this, go to “Extensions” and then select “Manage Extensions” from the browser menu, as shown in the picture below:
Chrome Browser > Extensions > Manage Extensions
After choosing Extensions, it redirects to the extensions page in Chrome. Make sure to enable the Developer mode here.
Once that’s done, you need to click the Load unpacked button that will allow us to load our project in the Chrome extension store.
Now, the extension is available in our Chrome extension store. You can also pin the extension in the browser as shown in the gif above:
This extension works only in your browser. If you want to publish it on the Chrome Web Store, you can follow this link.
📣 Conclusion
Finally, we’ve created a simple Chrome extension that allows users to enable the right-click and copy functions on websites where they’ve been disabled. Our straightforward guide also offers insights into the basic structure and development process of a Chrome extension.
Developing Chrome extensions is a fun way to bring your ideas to life and improve your browsing experience. Feel free to modify the Source Code of our extension and customize it to fit your specific needs. Happy coding!
WebEngine is an Open source Content Management System (CMS) for Mu Online servers. Our main goal is to provide a fast, secure and high quality framework for server owners to create and implement their own features to their websites.
Getting Started
These instructions will help you deploy your own website using WebEngine CMS.
Prerequisites
Here’s what you’ll need to run WebEngine CMS in your web server
As a stock Android user, I noticed that my device does not natively support a double tap to lock screen feature. So, I came up with an idea to create a lightweight app that locks my device with a single tap. My main goal was to make it look like a stock app and keep it lightweight.
Additionally, my device has a dedicated Google Assistant button that I don’t use. I decided to repurpose it to be useful by remapping it to toggle the device flashlight. Now, it functions as a flashlight toggle.
This C# API client for the UpCloud API provides a web service interface using HTTPS. It allows extensively featured resource management on UpCloud’s IaaS with easy to use functions. The API client follows the RESTful web service principles wherever possible.
The base URL for all API operations is https://api.upcloud.com/ and require basic authentication using UpCloud username and password. We recommend creating a subaccount dedicated for the API communication for security purposes. This allows you to restrict API access by servers, storages, and tags ensuring you will never accidentally affect critical systems.
It’s recommended to store the username and password as environmental variables while developing API applications to avoid accidentally publishing your account credentials.
baseAuth
Type: HTTP basic authentication
Username: Your UpCloud API username
Password: Your UpCloud API user’s password
Issues
Found a bug, have a problem using the client, or anything else about the library you would want to mention? Open a new issue here to get in contact.
License
This project is distributed under the MIT License, see LICENSE.txt for more information.
This C# API client for the UpCloud API provides a web service interface using HTTPS. It allows extensively featured resource management on UpCloud’s IaaS with easy to use functions. The API client follows the RESTful web service principles wherever possible.
The base URL for all API operations is https://api.upcloud.com/ and require basic authentication using UpCloud username and password. We recommend creating a subaccount dedicated for the API communication for security purposes. This allows you to restrict API access by servers, storages, and tags ensuring you will never accidentally affect critical systems.
It’s recommended to store the username and password as environmental variables while developing API applications to avoid accidentally publishing your account credentials.
baseAuth
Type: HTTP basic authentication
Username: Your UpCloud API username
Password: Your UpCloud API user’s password
Issues
Found a bug, have a problem using the client, or anything else about the library you would want to mention? Open a new issue here to get in contact.
License
This project is distributed under the MIT License, see LICENSE.txt for more information.
Support for WerwackFx’s StoryLiner – Blender Add-on
Attention, directors, storyboarders and animators! Introducing a groundbreaking real-time video editing tool that seamlessly blends 2D and 3D. Take your projects to the next level with our cutting-edge software. Unlock endless possibilities and unleash your creativity like never before. This is a game-changer tool, get your hands on it today and revolutionize your work!!
StoryLiner is a Blender add-on offering a full workflow and set of tools for storyboarding, previzualisation and storytelling. It is designed from the ground up to be used in production and to be artist-friendly.
In addition to all the drawing features coming to boost the Grease Pencil environment, StoryLiner introduces a true shot entity in Blender scenes, as well as a wide and powerful set of tools to build and edit sequences in real-time directly in the 3D context, in a non-linear way.
Current version is V 1.3.430 for Blender 4.3 and 4.4.
✨ See what’s new in the change log.
Generative Adversarial Nets (GANs) face problems when dealing with the tasks of generating discrete data. This non-differentiability problem can be addressed by using gradient estimators. In theory, the bias and variance of these estimators have been discussed, but there has not been much work done on testing them on GAN-Based Text Generation. We will be analyzing the bias and variance of two gradient estimators, Gumbel-Softmax and REBAR, on GAN-Based Text Generation. We propose two sets of experiments based on differing sentence length and vocabulary size to analyse bias and variance, respectively. In this work, we evaluate the bias and variance impact of the above two gradient estimators on GAN Text Generation problem. We also create a novel GAN-Based Text Generation model on top of RelGAN by replacing Gumbel-Softmax with REBAR. The performance of the new model is evaluated using BLEU score and compared with RelGAN.
RebarGAN has lower average bias than GumbelGAN for all the sequence lengths and vocabulary sizes we tested. However, at the same time, GumbelGAN has lower average log variance compared to RebarGAN for all tested values.
Rebar-Based RelGAN Evalution
We trained both RelGAN and ReLbarGAN on the Image COCO dataset with 5 MLE pretraining epochs, batch size of 16.
More details can be found in this report.
Cloud Bigtable is a datastore supported by Google for storing huge amounts
of data and maintaining very low read latency. The main drawback to using Bigtable is that Google does
not currently have an official asynchronous client. Within Spotify we have been using the RPC client which is
a pain to use. This library aims to fix that by making the most common interactions with Bigtable clean and easy
to use while not preventing you from doing anything you could do with the RPC client.
To give an example of using the base RPC client (which gives the BigtableSession object), this is how you would
request a single cell from Bigtable.
StringprojectId;
Stringzone;
Stringcluster;
BigtableSessionsession;
StringfullTableName = String.format("projects/%s/zones/%s/clusters/%s/tables/%s",
projectId,
zone,
cluster,
"table");
// Could also use a filter chain, but you can't actually set all the filters within the same RowFilter object// without a merge or chain of some sortfinalRowFilter.Builderfilter = RowFilter.newBuilder().setFamilyNameRegexFilter("column-family");
filter.mergeFrom(RowFilter.newBuilder().setColumnQualifierRegexFilter(ByteString.copyFromUtf8("column-1")).build());
filter.mergeFrom(RowFilter.newBuilder().setCellsPerColumnLimitFilter(1).build()); // By default it is 1finalReadRowsRequestreadRowsRequest = ReadRowsRequest.newBuilder()
.setTableName(fullTableName)
.setRowKey(ByteString.copyFromUtf8("row"))
.setNumRowsLimit(1)
.setFilter(filter.build())
.build();
finalListenableFuture<List<Row>> future = session.getDataClient().readRowsAsync(readRowsRequest);
finalListenableFuture<Cell> cell = FuturesExtra.syncTransform(future, rows -> {
// This doesnt actually check if the row, column family, and qualifier exist// IndexOutOfBoundsException might be thrownreturnrows.get(0).getFamilies(0).getColumns(0).getCells(0);
});
Bigtable Client
The goal of this client is to let you query what you want with minimal overhead (there should
be no need to create all these filter objects) as well as give you the object you want without
needing to constantly convert a list of rows down to a single cell. Note that these examples
use a String as a row key. Bigtable keys are really byte arrays. Strings in this api is just
a convenience. Under the cover the string “row” is converted to a ByteString. In reality
you should use byte arrays as keys as that will
be more efficient.
Here is the same query as above using this client wrapper.
StringprojectId;
Stringzone;
Stringcluster;
BigtableSessionsession;
Bigtablebigtable = newBigtable(session, projectId, zone, cluster);
finalListenableFuture<Optional<Cell>> cell = bigtable.read("table")
.row("row")
.column("family:qualifier") // specify both column family and column qualifier separated by colon
.latestCell()
.executeAsync();
Performing Reads
The goal of this client is to make the most tedious and common interactions with Bigtable as painless as possible.
Therefore reading data is an extremely large focus. Here are some examples of reading data.
Get full column family within row
finalListenableFuture<Optional<Family>> family = bigtable.read("table")
.row("row")
.family("family")
.executeAsync();
Get multiple columns within a row (Currently all need to be in the same column family but hopefully that gets fixed)
// Get the entire columnfinalListenableFuture<List<Column>> family = bigtable.read("table")
.row("row")
.family("family")
.columnQualifiers(Lists.newArrayList("qualifier-1", "qualifier-2"))
.executeAsync();
// Get the latest cell in each columnfinalListenableFuture<List<Column>> family = bigtable.read("table")
.row("row")
.family("family")
.columnQualifiers(Lists.newArrayList("qualifier1", "qualifier2"))
.latestCell()
.executeAsync();
Get columns within a single family and within column qualifier range
Get all rows between different ranges or with certain specific keys
(these functions add rows to the row set, instead of filtering)
finalListenableFuture<List<Row>> rows = bigtable.read("table")
.rows()
.addRowRangeOpen(myStartKeyOpen, myEndKeyOpen) // add an exclusive range
.addRowRangeClosed(myStartKeyClosed, myEndKeyClosed) // add an inclusive range
.addKeys(extraKeys) // add some keys you always want
.executeAsync();
Note that currently there is no half open, half closed range.
Other Operations
The client supports other Bigtable operations as well, with hopefully the rest of all possible operations coming
soon.
Mutations (Writes, Deletions)
Mutations are performed on the row level with many mutations possible within a single call. Mutations include
writing new values as well as deleting a column, column family, or an entire row and all data help in each.
ReadModifyWrite (Atomically Update or Append To A Column)
ReadModifyWrite is useful for either incrementing the latest cell within a column by a long or appending bytes
to the value. If the column is empty, is will write a new value. Once again this operation is on the row level
with multiple ReadModifyWrites possible in a single request.
Increment a couple counter columns and append a value to another
Perform a read and a set of mutations depending on whether the read returns data. This is not yet implemented but
here are some ideas on how this operation might be implemented in the future.
Have the check specified like a read, then allow mutations to be added.
Table and Cluster Admin Operations – NOT YET IMPLEMENTED
It is unclear whether there is a need this wrapper to provide the admin operations, though it would be pretty easy
to include.
How to Release
A local build and deploy is the easiest way to make a release at this point. For setup follow instructions in: scio instructions modified to work with maven.
For credentials to push to sonatype, create an access token with your sonatype login and place the access token
in your maven settings.xml as in:
server>
<!-- sonatype repository -->
<id>ossrh</id>
<username>access-token-name</username> <!-- access token tied to an account sonatype.org -->
<password>access-token-password</password>
</server>
mvn clean javadoc:jar source:jar deploy should do the rest.
Release deployments (from mvn deploy) will be put into a staging area at sonatype. Read the following to release the staged deployment how to release and maven.
With effort we could get automatic deployments via travis. The travis build console is here: travis.org. You’ll need access to travis.org (not .com) to access
the builds.
Open Problems and Questions
One problem is that currently it is not really possible to do queries for nested lists. For example there really is
no way to do a ColumnRange within a RowRange or request multiple columns within different column families.
Something like BigtableColumnsWithinFamilies could be added where it keeps track of needing different column
families but that is confusing. Another option is adding the filtering methods to every Read object which would also
be super confusing.
Code of conduct
This project adheres to the Open Code of Conduct. By participating, you are expected to honor this code.