dot42 - Moving Forward

If you have been following dot42 since we released the tech preview in  Jan 2013 or a bit later, then you also have noticed that activity dropped significantly since Sep 2013.

Since the release of dot42 we have seen great interest from the community. Our approach to writing Android applications in C# without requiring the Mono run-time was received positively.

There are a few fundamental problems with the design of dot42 though. Its primary function is converting compiled C# code (IL) to byte code that runs on Android (DEX). Maintaining and developing this kind of code requires highly specialized knowledge of both byte formats and compiler construction skills. But there are more downsides to the IL-to-DEX approach. I will discuss them and tell you how we plan to move forward.

TallComponents

First I need to diverge a bit. Besides being the owner of dot42, I am also the owner of TallComponents: a vendor of .NET SDKs for generation, manipulation and rendering of PDF documents. Recently, we decided to complement our .NET SDKs with Java counterparts. We hired a PhD candidate specialized in program transformations and cross-compilation. His first task was to decide how to do the conversion. Basically, there were two options: convert at byte level (from IL to Java byte code) or convert at source level (from C# to Java). He picked the latter and the ratio applies equally well to dot42.

Convert Byte Code vs Convert Source Code

Finding and fixing bugs

A bug in the byte converter often results in the VM throwing the infamous java.lang.VerifyError. This exception is 'thrown when the VM notices that an attempt is made to load a class which does not pass the class verification phase'. Finding the bug is like going through a patient's poo to infer what was wrong with his meal. A typical example is that our DEX runs fine inside the Dalvik but crashes inside ART. If you convert at source level, the Java compiler takes care of correct byte layout.

Experiment with conversion code

When working on the conversion of a particular C# language to Java, you can easily experiment with C# and Java source code before you add the technique to the conversion code. Working on a proof of concept is much harder with byte code.

Write vs Use a compiler

Compilers for different virtual machine models or CPU architectures use different optimizations. IL generated by the C# compiler targets the stack based CLR. The Dalvik VM on the other hand, uses a register based machine model. If we convert from IL, the C# compiler has already thrown away information that could have been valuable for the Java compiler. By converting C# to Java we allow the Java compiler to generate the most efficient Java byte code or DEX.

Roslyn

Now that Microsoft provides an open-source C# compiler with rich code analysis APIs through "Roslyn", we have a powerful instrument available to convert C# to Java.

Transparency

Finally, for the dot42 developer, the conversion of C# to Java will make the process much more transparent and debuggable. She will be able to take dot42's output and open it in Eclipse or Android Studio.

Moving forward

We will release the dot42 compiler tools under the GPLv2 license. Note that GPL does not propagate to the output of GPL software. You will be able to build closed source Android applications with dot42 (including commercial). The run-time code is already licensed under Apache License 2.0: https://github.com/dot42/api. You will no longer need a Professional license to develop commercial applications with dot42. The next update of dot42 will not require activation.

Anyone can fork the dot42 project and build further on the IL to DEX approach. From now on, we will invest in replacing the IL to DEX cross compiler with a C# to Java converter. This effort will be sponsored by TallComponents. The code will also be released under the GPLv2 license. Any run-time code will be licensed under Apache License 2.0.

We are currently preparing the source code for release (removing 3rd party references, removing obfuscation, adding build instructions, etc.). We will let you know as soon as the source code is available on GitHub.

1.0.1.81 - What's new

1.0.1.81 - What's new

See also changelog. Since we are now keeping track of all issues on GitHub, you can also visit the milestone that corresponds to this release.

To get the latest version, run "Check for updates" from the Device Center or download.

Fixed: Several import JAR issues

When we import a JAR, we generate a C# proxy. Java allows constructs that are not supported by C# and vice versa. Current issues are mostly related to choosing alternative mappings. We fixed import issues with Ksoap2-android, usb-serial-for-android and ormlite and gdx-backend-android.jar (libgdx).

New: Invoke apkspy from command line.

You can invoke apkspy (to read JAR or APK) from the command line. A time saver when debugging.

Fixed: System.BitConverter did not respect endianess.

See GitHub issue.

Fixed: Several System.String.Format issues

See GitHub issue.

Fixed: Unsubscribing from event fails

See GitHub issue and discussion on StackOverflow.

1.0.1.78-1.0.1.80 - What's new

1.0.1.78-1.0.1.80 - What's new

Check the changelog for the detailed list of changes.

To get the latest version, run "Check for updates" from the Device Center or download.

New: Added support for Android 4.4 API

Changed: Improved not-activated message

After installing dot42, the Device Center prompts you to activate dot42 by entering your serial. If, after installation, you do not start the Device Center but create or open a new dot42 project in Visual Studio, then the build fails with the message: "This certificate cannot be used with your current license.". This left many users puzzled with no hint how to solve this. We have changed this message to be more descriptive as follows: "Build failed because dot42 has not been activated. Start the dot42 Device Center to activate.".

Fixed: Layout of activation dialog messed up

In some occasions, the layout of the activation dialog is messed up, making it practically impossible to enter the serial.

Changed: Free Apps key no longer required for publishing

Prior to this version, with the Community License, it was not possible to sign your APK with a certificate that has a lifetime longer than two years unless you registered a so-called Free Apps key. This key had to be requested online by providing the name and MD5 fingerprint of your APK. 

As of this version, you no longer need a Free Apps key. You can use a certificate with any lifetime with your Community License. Consequently, you no longer require a Free Apps key to publish your APK to the Play Store. Note that this is not a legal change - just a technical one to reduce the clutter.

Fixed: Improved "Unknown serial message"

This dreaded message was shown if you tried to activate a serial that had already been activated. 

New: Added support for AndroidManifest.xml

See blog post for details.

AndroidManifest.xml Support Added

At the very start we decided to replace the traditional AndroidManifest.xml with C# attrbutes. The reasoning was that this is what C# developers are more familiar with. E.g. the following:

[assembly: Application("dot42Application2")]

translates into:

<application android:label="dot42Application2">

Under the hood, the dot42 compiler generates the AndroidManifest.xml from the attributes before packaging it with the APK.

User feedback told us that something that was intended to be a convenience was more of an obstruction.

Manifest resource item

With 1.0.1.80 we have introduced the manifest resource item template. This allows you to include a traditional AndroidManifest.xml in your dot42 project.

You can either create a new one from the Solution Explorer like this:

• from the context menu of your project select Add -> New Item... 
• From the dialog, select node dot42->Android
• From the available items select Manifest resource and set its name at the bottom of the dialog

Visual Studio will now add an Android Manifest to your solution. Some information is taken from your project settings, and inserted in the manifest.

Or you can simply copy an existing manifest to your project, include it in your project and set the build action to "ManifestResource" like this:

If you compile your project, Visual Studio will now detect that you have a manifest in your project and uses this file instead of generating one from attributes. This gives you more freedom over what your manifest contains. But with more freedom comes more responsibility: you have to make sure that all information in the manifest is correct. The only thing we check is if the SDK versions of your project match those in the manifest file. If not, a build error is generated. Clicking this error will take you to the attribute in the manifest that needs to be changed.

If you have both manifest attributes and a manifest file, then your attributes are ignored.

You can have only one manifest in your project. If more are found, a build error is generated for each manifest file. This makes it easier to locate all manifest files.

What about my 'old' manifest attributes?

These will continue to work. Your manifest attributes will only be ignored if you add a manifest resource to your project. If you want to transition from manifest attributes to an explicit manifest file, you can find the generated AndroidManifest.xml in \obj\Debug\TempCode after a successful build. Copy it to the root of your project and include it as described above.

dot42's .NET implementation soon to be released under Apache 2.0

Due to its design, dot42 supports the entire Android API and part of the .NET API. The .NET classes are handcrafted on top of the Android/Java API. Shortly, we will be moving the .NET API implementation to github. It will be licensed under the Apache License, version 2.0. Note that this does not apply to the tools themselves.

This should allow you and others to help us extend the supported subset of the .NET API. Initially, the repository will be private and a select group will be allowed access. After things ironed out, it becomes public. Should not take longet than a couple of weeks. Anyone is invited to request early access (frank at dot42.com).

1.0.1.77 - What's new

To get the latest version, run "Check for updates" from the Device Center or download.

New: Added CancellationToken support to async.

Added CancellationToken class. Also added many methods overloads that use this type.

New: Added refresh function to Device center.

You can now scan for new devices from the Device Center.

New: Added copy to clipboard function to Assembly Checker.

If you encounter missing types or members, use this function to copy everything to the clipboard and paste it to an email for support.

New: 'Import res folder' in VisualStudio

The Android SDK requires a folder structure for your resources under a folder called "res". In dot42 you can put your resources anywhere in your project, and the filename contains resource qualifiers. You can now  import a traditional Android "res" folder into your dot42 project and all the conversions are done automatically.

Changed: Activation is started directly from installer.

At the end of the dot42 installation you will be asked to activate your license.

Changed: Removed support for creating/editing emulator images.

All support for creating and editing emulator images has been removed. We expect developers to either run a real device or use a virtual machine instead.

Changed: Replaced device connection wizard with online help.

No comments

Changed: Various UI improvements.

No comments

Changed: AtomicInteger/AtomicLong had a few incorrect properties.

The properties AndIncrement and AndDecrement have been removed.

Fixed: DateTime.ToString/ToShortDateString

Some format strings yielded incorrect  results.

Fixed: Add JAR reference missing from Visual Studio.

The "Add Jar reference" function was broken.

1.0.1.75 - What's new

To get the latest version, run "Check for updates" from the Device Center or download.

This release does not contain any new features. It fixes several issues found in the 1.0.1.74 release.

Fixed: Async on single CPU systems

The implementation of async may throw an exception or block execution on single CPU systems.

Fixed: Nested interface constants classes could be non-visible

Nested interface constants classes (e.g. ContactsContract.ISyncColumnsConstants) can be nested protected and as such not visible to you as a programmer. Now all such classes are public.

Fixed: Nested try/catch blocks with catch(object) are incorrect

Sometimes the C# compiler generates a try/catch block with a catch(System.Object) handler. If you combine that with a catch(System.Exception) handler, the order of catch handlers becomes invalid.
Now all catch(System.Object) handlers are compiled as catch-all handler, solving the order issue.

Fixed: Some local variables do not appear in Locals window

In the Locals windows of the debugger, some local variables of large methods do not appear as expected. This was related to register spilling.