Context issue when using Anko DSL within customized adapter for ListView

I am trying to write a general-purpose adapter for ListView which allows using Anko DSL as the content of items. The code is listed below. As you can see, there is an ugly patch with(viewGroup!!.context) to make the code work. It doesn’t like other Anko example you have see. If I remove the with statement, my app will is crash by

FATAL EXCEPTION: java.lang.ClassCastException: android.widget.FrameLayout$LayoutParams cannot be cast to android.widget.AbsListView$LayoutParams.

Is there any way to avoid the with statement?

import java.util.*
import android.view.View
import android.view.ViewGroup
import android.widget.BaseAdapter

class AnkoAdapter(itemFactory: () -> AbstractList<Any>, viewFactory: (index: Int, items: AbstractList<Any>, view: View?, viewGroup: ViewGroup?) -> View): BaseAdapter() {
    val viewFactory = viewFactory
    val items: AbstractList<Any> by lazy { itemFactory() }

    override fun getView(index: Int, view: View?, viewGroup: ViewGroup?): View {
        return viewFactory(index, items, view, viewGroup)
    }

    override fun getCount(): Int {
        return items.size
    }

    override fun getItem(index: Int): Any {
        return items.get(index)
    }

    override fun getItemId(index: Int): Long {
        return items.get(index).hashCode().toLong() + (index.toLong() * Int.MAX_VALUE)
    }
}

// -------------
// In main acitivty.
...
val items = listOf<String>("Mary", "Lisa", "Cheryl", "Linda")
val buttonCaption = "..."
listView.adapter = AnkoAdapter({items as AbstractList<Any>}) {
    index: Int, items: AbstractList<Any>, view: View?, viewGroup: ViewGroup? ->
    with(viewGroup!!.context) {
        linearLayout {
            textView(items[index].toString())
            button(buttonCaption)
        }
    }
}

Just watch this video. It provides the answer.
Here are modified code:

class AnkoAdapter<T>(itemFactory: () -> List<T>, viewFactory: Context.(index: Int, items: List<T>, view: View?) -> View): BaseAdapter() {
    val viewFactory = viewFactory
    val items: List<T> by lazy { itemFactory() }

    override fun getView(index: Int, view: View?, viewGroup: ViewGroup?): View {
        return viewGroup!!.context.viewFactory(index, items, view)
    }

    override fun getCount(): Int {
        return items.size
    }

    override fun getItem(index: Int): T {
        return items.get(index)
    }

    override fun getItemId(index: Int): Long {

        return (items.get(index) as Any).hashCode().toLong() + (index.toLong() * Int.MAX_VALUE)
    }
}

val items = listOf<String>("Mary", "Lisa", "Cheryl", "Linda")
val buttonCaption = "..."
lockView.adapter = AnkoAdapter<String>({ items }) {
    index, items, view ->
    linearLayout {
        textView(items[index])
        button(buttonCaption)
    }
}